The 2018 State of DevOps Report from DORA was released last week. This is the first one I’ve read all the way through (failure on my part!), as opposed to just skimming the highlights.. and there are some very interesting things to be found inside.
Being better at shipping software is being better at business.
This feels true in the filter bubble of startups and Silicon Valley. But going back to the report of 2014, the data shows that high performers at shipping code have at least 50% better odds at beating their one business goals, like rev targets, than low performers.
Our analysis shows that elite performers are 1.53 times more likely to meet or exceed their goals for organizational performance, and high performers are 1.38 times more likely to meet or exceed their goals. — 2018
This year we looked at both financial and non-financial measures of organizational performance. We found that high performers were twice as likely to achieve their own reported goals across both financial and non-financial measures. — 2017
In the 2014 DevOps survey, just over 1,000 respondents voluntarily identified the companies they worked for. Of these, 355 were publicly traded. We analyzed three-year results for these companies and found that they all outperformed the S&P 500 over the same three-year period. Of this group, those with high-performing IT teams, as identified by our analysis, had 50 percent higher market capitalization over three years than the public companies with lower-performing IT teams. The companies with high-performing IT teams were also twice as likely to have exceeded their own goals for profitability, market share and productivity. — 2016
This year, we hoped to validate this preliminary finding, but our sample size of stock ticker symbols was too small to conduct a meaningful analysis. However, we did find that this year’s high performers were 1.5 times more likely than their peers to exceed their organization’s’ profitability, market share and productivity goals, compared to last year’s high performers, which were 1.9 times more likely to exceed goals. — 2015
("Elite” performers vs low performers)
In 2016 low performers deploying every one to six months.
In 2017 they were deploying at least once per month.
In 2018 some cohort got fast but shy of on-demand fast, while the low performance bar was stable.
What do you call teams that today are slower than low?
In the 2018 report, better availability is explicitly correlated with high performance. And high performance teams are those that deploy faster. Those same teams also recover faster and have smaller lead times from dev to prod.
Analysis showed the availability measures are significantly correlated with software delivery performance profiles, and elite and high performers consistently reported superior availability, with elite performers being 3.55 times more likely to have strong availability practices.
High performers do significantly less manual work across all vectors, spend more time doing new work
This is toil. Reading from the book of Site Reliability Engineering [Google]:
Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows
High performers uniformly minimize or outright eliminate toil. And remember, high performers also are twice as likely to outperform their against their own business goals (revenue, profitability, etc).
I’m amongst the first to be skeptical about anything coming out of FANG [etc] companies being applicable to anyone else. But this is a clear case where it’s not just true for Google; it is also true for all of us.
Because [high performers] build quality in, they spend less time fixing problems downstream, freeing up more time to do value-add work.
This one is a truism. And yet, it’s always sacrificed to the gods of false promises your boss made to their boss made to their boss made to…investors. Break the cycle.
Overhead is Real
At the “elite” level the best case scenario for doing “new work”, aka creating business value, is 50% of the time. This means that the highest performers have 50% overhead. FIFTY PERCENT.
Considering the anecdata I hear from many engineering leaders, we’re all lying to ourselves and each other.
Something super interesting happens between the low and high performers. We get stuck.
Medium performers have the highest amount of manual work on all dimensions.
This same pattern happens in startup go to market growth, hype cycles, or becoming an expert in any field of endeavor.
In my experience, very few organizations want to do that kind of work or create the culture and incentives necessary for it. It’s not sexy. It’s not motivating. There aren’t quick wins anymore. It’s mostly manual labor.
This, to me, is the most interesting point from the report precisely because it’s such a common pattern that crosses disciplines. The only answer is: do the effing work.
Create your free account to unlock your custom reading experience.