What is the Best Way to Measure Developer Productivity in 2022?

What exactly are we measuring when we assess developer productivity?

Is it the number of lines written per working day? The minutes (or hours and days) between an impairment to service and a hotfix rolling out? Or maybe it’s the number of times a developer checks their posture during a standup.

Whatever you are measuring when looking at either your own productivity or that of your team – and however you are measuring it – it might be time to expand the metrics a little. Measuring productivity is essential in making improvements to software development and delivery, but we believe there is more to developer productivity – and crucially sustainability – than simply the speed at which they write their code.

What does Developer Productivity mean?

A productive and successful developer is one who writes code quickly and accurately; at least, this is the currently accepted industry standard. And for good reason. Of course speed and accuracy benefit the team and the organization at large: lead times remain low, change failure rates stay minimal, and software delivery is on time and of high quality.

The current paradigm of measuring developer productivity is based on a framework of four key software delivery performance metrics. By collating years of survey data from thousands of DevOps professionals, Google’s DevOps Research and Assessment team (DORA) were able to identify these four crucial benchmarks to measure developer productivity. This allows teams to define their output between [Elite and Low**]**, and flag the areas they need to push in to drive further success.

The four Accelerate metrics used to measure productivity:

1. Deployment frequency:

This is a measure of how often your team is successfully deploying code to production. As you would expect Elite teams deploy the most frequently – multiple times a day in fact – but the standard rate is around once per week.

2. Lead time for changes:

As well as how often code is deployed, it’s important to measure the pace of delivery. How much time does it take for committed code to reach production? More efficient teams report shorter lead times: less than one day for Elite performers, with the standard being one week.

3. Change failure rate:

What percentage of changes to code result in deployment failures or bugs requiring patches, rollbacks or other hands-on fixes? Elite teams report a rate of 0% – 15%, whilst in the most recent 2021 State of DevOps report the standard for High and Medium teams increased to 16% – 30%.

4. Time to restore services

This final metric is used to measure the time it takes to restore normal services in the event of incidents that impair users. Faced with unplanned outages or incidents, Elite teams recover services within an hour, whilst the standard time is around a full day.

Why measure DORA metrics?

The DORA metrics take a relatively holistic approach to productivity. Measuring both deployment frequency and lead time helps teams to identify if they are reaching their internal goals and their organization’s reliability targets, or if they need to retool aspects of their development cycle to make it more efficient. Combined with an accurate appraisal of change failure rate and time to restore services, these metrics measure stability, quality and tempo of developer work, providing teams with a clearer picture of overall productivity.

Tools like Haystack and Linear B help to measure the above metrics by providing constant analysis of your team’s performance, pulling data from streams including GitHub and GitLab to help you spot blockers in your software delivery performance, and make sure your team is efficiently hitting all four metrics.

But are there limitations to the DORA framework?

What DORA doesn’t tell you

What these four metrics fail to quantify is that even the single best developer in the world has a flaw they share with all the others: they are only human.

Big, fast numbers in relation to the DORA metrics are fantastic until they drop off for no clear reason, or if you find your team excelling in lead times but falling seriously short in restoring service.

If you are neglecting to measure the person as a whole then you’ll inevitably miss the reason for inconsistent productivity: there is no sustainability in measuring productivity and code-centric metrics alone.

Collaboration and Wellbeing

Real people can’t be measured purely by output; they need well-being checks to put the brakes on burnout, a work-life balance so they can thrive in both, and they need a positive culture of open collaboration to dedicate their full selves to work.

In DORA’s own State of DevOps Report 2021, they reported that teams sharing a culture of encouraging “information flow, trust, innovation, and risk-sharing” were more likely to report better software delivery performance and that valuing psychological safety was indicative of a high performing team.

If you’re looking to make developer success and productivity truly sustainable, you’ll need to expand your metrics to include both wellbeing and collaboration as measurable data.

Luckily, we’ve got the tool for that.

How do we do it?

The Adadot development team does things a bit differently. We combine code-based software delivery metrics with person-based wellbeing and collaboration data, to make developer success sustainable.

We pull productivity data from platforms including GitLab, Jira and Google Workspace, and combine these with data from collaboration tools like Slack to give us big-picture insight into our team’s software delivery performance.

If you are part of a tech team you should be looking at the following:

• How are you managing your work-life balance?

• How many times have you replied to a Slack message well into the evening?

• Are you forging strong working relationships by collaborating with the same developers on multiple projects?

  
  

By looking at well-being and Collaboration alongside Work metrics, being conscious of behaviors including out-of-hours communication, responsiveness, and time spent on collaborative coding projects, you can be sure that you are on your way in your journey towards sustainable development success.

We are consciously improving our own habits, including cutting down on meeting times for focused work, reducing excessive fragmentation across multiple projects, and monitoring out-of-hours work to mitigate burn-out, helping you ensure that your success and productivity are viable, healthy and long-term.

We aren’t replacing the DORA metrics, we’re just making them more human.

Also Published here