How to measure developer productivity

What is developer productivity?

Developer productivity usually refers to the effectiveness and efficiency of teams at creating high-quality software that provides significant business value within a pre-established timeline. It can also refer to the proficiency and efficiency of individual team members; however, it is the combined productivity of the team members that is most important to the SDLC.

Benefits and challenges of measuring developer productivity

Before beginning any measurement initiative, it is important to clarify what it is that teams are hoping to learn from the metrics and how they will use them to improve. Informed release planning and continuous improvement are the most important benefits that an organization can derive from measuring developer productivity at the project and team levels. Measuring individual developer productivity is about measuring developer performance. Teams can use individual data to learn about each developer's strengths and weaknesses and to balance the engineering team's skill set. Aggregate team productivity measurement can help establish timelines and plan releases.

Focus not on the measurements themselves, but on how they tell the story of the developer experience.

There are many challenges to measuring developer productivity. First, it is difficult to measure a developer's productivity based on their various work tasks, which have both objective and subjective aspects without a predominance of either. The nature of development is creative and involves multiple different kinds of tasks, so it is difficult to apply objective measurements in every case.

For example, tracking developer productivity based on lines of code written does not measure everything a developer does and won't cover aspects of code such as level of code quality or technical debt. Additionally, metrics that could be useful to track team performance are not necessarily useful at the individual level.

How to measure developer productivity

When attempting to measure developer productivity, it is critical to take a holistic approach, focusing not only on the objective output measures, but also considering the more subjective aspects of the roles. This is difficult because, to have a reasonable level of accuracy, it requires multiple types of measurements, some of which can be quite difficult to quantify.

Some organizations approach this conundrum by looking at developer experience (DevEx) rather than focusing on objective developer productivity. DevEx is an approach to measure developers' work experience, i.e., how developers feel about their day-to-day work and the issues that affect their output. It involves more subjective, qualitative aspects including collaboration, culture and cognitive load, which increases with frequent context switching.

How not to measure developer productivity

Contrary to what some might believe, developer productivity is most effective when measured based on team versus individual performance. Tracking individual productivity has its place in performance management and is best used as a tool for improvement. Individual productivity measures that highlight areas of improvement can work in team resource planning to ensure the team has the right types of expertise.

However, measuring individual programmer activity can affect team morale, potentially limit collaboration and reduce the team's overall cohesiveness. These impacts will likely reduce teamwork and productivity rather than improve them.

5 steps to measure developer productivity effectively

The steps to measure developer productivity are as follows:

1. Determine how to use the measurements

Clarify the goals of measuring software developer productivity. Some goals might be to better understand bottlenecks, reduce burnout or improve deployment frequency. This understanding is critical as it will lead to the correct approach and point to relevant metrics.

2. Develop a measurement strategy

This is the plan for what types of metrics the team will collect and how the team will manage their collection and use. The strategy should be as follows:

1. Transparent. Define who has access to performance data within the strategy and how data informs decision-making.
2. Tailored to the team's context. Ensure the defined strategy is tailored to the team's overall maturity, its tech stack and delivery model.
3. A balance of quantitative and qualitative measurements. Quantitative metrics might include deployment frequency, number of pull requests or codebase complexity. Qualitative measurements might include perceived ease of software delivery and employee engagement or satisfaction. Teams might use elements such as a dashboard template for deployment cycle times or a quarterly, monitored DevEx survey.

3. Select the actual developer productivity metrics

Some of the most popular measurement frameworks are as follows:

1. DORA. Focuses on measuring DevOps processes, including deployment frequency, lead time to completion, change failure rates and mean time to restore service. DORA stands for DevOps research and assessment metrics. DORA metrics are especially suited to teams using CI/CD or DevOps methodologies that want to focus on software delivery optimization.
2. SPACE. Tracks dimensions such as satisfaction and well-being; performance; activity; communication and collaboration; and efficiency and flow. Specific SPACE metrics might include number of deploys, code review time or incident response participation.
3. Value stream metrics. Uses value stream metrics to track the flow of units of development. These help teams visualize the flow of work from idea to delivery and look at elements such as handoff and delays, and value-added time -- time spent on activities that directly contribute value.
4. Flow metrics. Evaluates the efficiency and effectiveness of the progress of work through a system. Some examples include flow efficiency -- the ratio of active work time to total process time -- and flow load -- the amount of work in progress. Story point estimation could also work as an option to measure flow velocity -- how many work items are completed over time.
5. Developer Velocity Index. Measures Agile teams' capacity to deliver high-quality features quickly. The Developer Velocity Index is best suited as a high-level benchmark for engineering productivity across different business units. It focuses on developer tool usage and technology adoption to gauge how well the work environment supports fast, quality delivery.
6. Talent capability score. Measures leadership and management potential. This can help leadership teams align engineering metrics with talent development and help teams allocate talent appropriately according to project requirements.
7. DevEx metrics. Measures developer perceptions of tools, workflows and culture. DevEx metrics can examine cognitive load, onboarding experience and collaboration quality through surveys, sentiment analysis, developer journey mapping and internal net promoter score.

4. Introduce the measurement strategy

Teams should understand why metrics will be collected and how they will be used. Neglecting this step will negatively affect the team morale, which negatively affects productivity. Make the goals of the metrics clear to the team.

5. Review the results and fine-tune the approach as necessary

Use contribution analysis to measure how various types of initiatives affect productivity. Involve developers in strategy refinement, perhaps starting with a small pilot team and expanding as the strategy takes shape.

The most important aspect of measuring developer productivity is to use those measurements with a goal of continuous improvement. Focus not on the key metrics themselves, but on how they tell the story of the developer experience. Using that story to make adjustments that will improve their work environment is the key to fostering and supporting productive developers.

Gerie Owen is a QA engineering manager at Roobrik. She is a conference presenter and author on technology and testing topics, and a certified Scrum master.