Tracking Pull Requests with Open-source

The Idea

At a previous company where Anton worked, regular code review was tracked using a GitHub Action. When a PR was created, it analyzed all previous PRs in the repository, calculated statistics, and added them as a comment: number of code reviews, comments, and average review time.

However, the workflow and lack of configuration options often failed to provide accurate stats. For example, if one developer created a PR at night and another reviewed it in the morning, the "average review time" would inaccurately show as eight hours. Additionally, some PRs to the main branch didn’t need tracking. The stats were only visible when the PR was created, so to get current data, you had to open a new PR.

These limitations inspired Anton to develop his own solution. His first implementations included real-time stats and work-hour settings for more precise calculations. The team loved it, and soon they switched to PullKeeper for their tracking needs.

Over time, PullKeeper gained new metrics and settings. Now, users can configure work hours, vacations, and weekends for both the company and individual developers. The solution can evaluate comment quality and include it in the stats. And, importantly, PullKeeper works not only with GitHub but also with GitLab, with plans to expand to other platforms. As features expanded, the interface evolved as well. Anton also switched the frontend from React to SolidJS for faster performance.

Currently, 32 companies are using PullKeeper to track 84 repositories.

Key Features

• Reviewer Statistics: Data is updated hourly, tracking reviews, comments, average review time, and more. Stats can be collected for individual repositories or across the entire company.

• Security: The app only requires read-only permissions for PR access and doesn’t access any code. Developers can view their stats through an easy authentication method, though this feature can be disabled if desired.

• Time Calculations: Multiple configuration settings are available for more accurate average review time calculations.

• Exclusions: Ability to exclude specific team members (e.g., bots or testers) and certain PRs (e.g., those in the main branch) from the stats.

• Data Comparisons: Historical data is saved and can be compared with current periods to assess progress.

• Notifications: Send updates to Slack, Discord, or Telegram with the latest news on companies and repositories. Data can also be sent to a custom URL or accessed via API.

• GitHub and GitLab API Integration: Reviewer, PR, and comment data are collected through the GitHub or GitLab API.

• Achievements: An achievements system helps track and compare developer progress.

• Comment Analysis: Comments are evaluated for relevance according to specific guidelines. Quality comments positively impact statistics.

Installation

All project code is available for review. You can install it locally or on your own server. To launch the project locally:

$ bundle install
$ rails db:create
$ rails db:schema:load
$ rails db:seed
$ yarn install
$ EDITOR=vim rails credentials:edit

Setup

1. Add a Company: This is an abstraction for a group of repositories with similar configurations.

2. Add a Repository: Provide the repository link for GitHub or GitLab, which will then be used to pull statistics. Only one user needs access to pull data; others can view the stats.

3. Add an Access Token: Add a read-only token. PullKeeper retrieves timestamps for PRs, comments, reviews, and authors.

Done! Developer data is collected at the repository level and aggregated for the entire company. Each repository also has additional metrics with specific stats. To access collected data, developers just need to log in to PullKeeper.

Additional Settings: Privacy, work hours, exclusion rules for pull requests, various insight calculation options, and notification configurations.

Pricing

The basic version of PullKeeper is free, but additional options are available for larger companies. The basic version includes:

• Eight developer metrics
• 16 repository metrics
• Tracking for five repositories
• Company-level metrics based on repositories
• Easy access for developers
• Work hours and vacation configuration
• Exclusion rule settings
• Analytics delivery to Slack, Discord, or Telegram
• API access
• Achievements system

The premium version includes all the above, plus 15 developer metrics, unlimited repositories, data comparison with previous periods, priority support, new feature implementation, and onboarding support. The premium package costs €25 per month or €250 per year, with a 100-day free trial available. An enterprise version with self-hosting and desktop app options is also available.

Future Plans

PullKeeper continues to evolve. The project’s author is working on a full-featured desktop app for enterprise clients, which runs locally without a server.

Updates are also planned for the web app, including a chat feature for admins, karma (achievements over time), and enhanced notifications—such as expired token alerts, regular updates, and invitation emails for new users. Azure DevOps integration will be added soon.