The Most Important Enginnering KPIs for Your Board Deck

Preparing for a board meeting is a stressful experience for everyone. For many CTOs, it’s also an exercise in futility, trying to zero in on engineering KPIs that accurately represent everything that’s happened in the department. The information that usually makes it to the board deck — information on completed features and incident reports — doesn’t tell the whole story.

A list of features shows what the team accomplished, but doesn’t reveal anything about how the team works. It can’t reveal process issues that need to be addressed or highlight challenges the team has overcome. Similarly, incident reports might illuminate problems in the codebase, but they’re devoid of context about how the team is dealing with those issues. Other departments have self-evident, objective KPIs. Sales, for example, can walk through their pipeline from demos booked to contracts signed, using objective measurements to chart their department’s progress over time.

Engineering leaders, on the other hand, must get creative to provide that level of clarity.

You’ll need to think carefully about the story you want your slides to tell, then choose engineering KPIs that map to those specific insights. When considering a metric, we recommend asking:

• Will this demonstrate the department’s success (or lack thereof)?
• Does this highlight something interesting or unexpected, and start a conversation?
• Will this help me ask for advice from the board?

These parameters will help you select the right data to drive productive conversations, so you can walk away with actionable guidance from the board.

Start with Engineering Success Metrics

To convey big-picture information, start your presentation with engineering success metrics. Success metrics speak to outcomes; they measure the results your team is getting.

In engineering, there are two aspects of success: planning and execution. Engineering decks usually include feature lists, which speak to your team’s ability to plan. But most engineering decks stop there. To dig into execution — your team’s ability to consistently ship projects — you’ll need another set of success metrics.

When choosing success metrics that highlight execution, look for metrics based on objective data rather than subjective measurements like story points. With objective data, you’ll be able to track progress across teams, projects, and quarters, and set more effective goals for your department.

Use metrics like:

• Deploy Volume - A count of the engineering team’s deploys within a given period of time, Deploy Volume can help quantify the engineering team’s capacity.
• PR Throughput - The number of Pull Requests merged over a period of time. Similar to Deploy Volume, PR Throughput can help you quantify how much the engineering team is getting done.
• Cycle Time - The time between the first commit logged in a Pull Request, and when that Pull Request is merged. Cycle Time acts as a proxy for engineering speed and can help you communicate how quickly the team is moving.

Drill Down with Revealing Engineering KPIs

After you report on success metrics, you’ll want to dive into additional metrics that provide critical context and perspective. Think about what you know to be true for your team, and then pick the data that will help you illustrate that to the board. You might want to include information that reveals the impact of a key decision or highlight a specific data point that raises an important question for discussion. Be open to all kinds of data — both quantitative and qualitative — and consider pairing multiple metrics to add dimension, or to answer questions before they arise.

To determine which additional metrics might be useful, ask yourself:

• What did we promise the board at our last meeting, and how can we represent whether we made good on that promise?
• What took up a lot of engineering time this quarter? For example, were you making a big security push, focused on new features, or working through technical debt?
• What big changes or strategic decisions did we make this quarter? For example, did you increase headcount, initiate a re-org, or change an aspect of your review process?

Your answers will be different every quarter, and they won’t always be noteworthy enough for the board. But your line of questioning might also uncover something that is important to share and not already covered in the deck.

For example, let’s say at last quarter’s board meeting, you discussed implementing a support engineering rotation. This quarter, you might want to show the board that it’s having a positive impact on the team’s ability to resolve customer issues. To do that, you could report on the Issue Cycle Time of Escalations, which measures how long it takes for an escalation to be addressed once it’s opened. Though highly specific, this software development metric would demonstrate the success of the support initiative.

Put Engineering Metrics in Conversation

Putting metrics in conversation with each other can also help paint a more complete picture of what’s happening on your team. For example, it might be useful to share the engineering department’s PR Throughput, coupled with incident data. Either one of these metrics would be useful in isolation, but taken alone, a high PR Throughput might raise the implicit question: is the engineering team sacrificing quality in order to keep deployment high? By pairing PR Throughput with incident data, you’d be able to demonstrate that the team isn’t solely focused on deployments, they’re also consistently delivering quality code.

You might also consider pairing metrics to demonstrate the impact of a strategic decision. Let’s say you recently hired three new engineers. You might want to share PR Throughput, paired with Active Contributors, to illustrate the team’s increasing output as the newly onboarded engineers begin to contribute more and more to the codebase.

Most board meetings include a lot of anecdotal evidence from the engineering department. The inclusion of objective metrics can help you show the board exactly what’s happening on your team, so they can participate in the conversation more effectively, and offer up more useful guidance.

Make Board Meetings Work for You

The goal of a board meeting is to provide vital context so that the board can do what it’s there to do — offer advice and help with key strategic decisions.

Subjective metrics and flawed measurements might feel like relevant updates, but they won’t reveal enough actionable information about what’s going on in your department. Think about what you want to get out of the board meeting, and then choose the data that provides essential context for those conversations.

Previously published at https://codeclimate.com/blog/engineering-kpis-board-deck/