@johncutlefish's blog

I am currently writing weekly here and have all my 2020 posts here.

Feedback Loops and “Done”

Published: March 13, 2017

1 7bPqmYF457WD2gRRsDX nQ Some teams just assume that fast feedback is impossible. It is more like build, build, build — maybe measure — learn eventually. Perhaps they deliver an enterprise product, and the thought of “experimenting” on their customers feels too risky and expensive. Or maybe they lack the tooling, autonomy, and schedule flexibility. The end result is that it feels very inefficient to “wait around” after the “project is complete”. So it is on to the next “project”, and the business (hopefully)

commits to monitoring the impact of the work and prioritizing follow-up work should it be required.

1 KmGlOcCdRELK0a0kQEW1lA

This is where Agile — even when perfectly implemented on the delivery side — fails to address the bigger picture of fast feedback and iterative development. Imagine if someone said “and now we are going to have a five month sprint to get feedback” … the team would cringe! Yet that is exactly what happens when you ship work into the abyss. We add complexity without validating that added complexity. Yes, in theory you are delivering customer-facing value with each sprint…but are you getting feedback with each sprint?

When teams talk about velocity, they rarely talk about the velocity of learning and/or feedback. If you think about lead time as the time from idea conception, to the customer “accepting the order” (or the work adding the desired value), you’ll typically see that the touch time for development is just a small fraction of the overall cycle.

1 y37ClzB4RNwhgmOQEMY41A Let’s say that a CEO dreams up a silver bullet in January, teams obsess over planning and research between February and May, the dev team works on the idea between May and June (while doing three other things in parallel)

, and ships it July 1st. Six months later, the response from customers is tepid, and it looks like the work missed its mark. In terms of lead time, the feature is still pending. It has been a year and one month, and it is still “In Progress”.

Since developer time is theoretically the limiter, what companies do is work hard to land work on this precious resource in the most efficient manner possible. They build incentives around this. They create department goals around this. They build complex dashboards and project plans around this. They obsess about relative prioritization. They enlist less precious resources to prepare the work beforehand.

If there’s one thing that the feedback to my feature factory post taught me, it is that organizations optimize around delivery (not validation) in very deep and complex ways. The approach becomes deeply rooted in the company culture, including how it celebrates and rewards people. The company literally runs in execution mode until things stop working, heads roll, they re-org, and the process repeats itself.

And it is not just individual organizations. Whole industries exist to sell ways to “get it right” before shipping, and make sure teams “make their commitments”.

There are, of course, powerful ways to reduce risk early in the product development cycle without pushing features live and measuring impact. This post is by no means an attack on those methods. Even still, few plans survive first contact with the enemy. Probably more important is that without feedback, teams are not able to meaningfully iterate and knock off any low hanging fruit that emerges from “first contact”. Months later the context has changed, the research is stale, and the cost of reengaging is high. But this approach feels like the economically optimal decision given perceived constraints.

1 qUSb3j0w6J43oVPSwC9xaA

And that’s the point of this post … to encourage you to question your constraints.

  • Are your constraints around customer feedback set in stone? Are they non-negotiable?
  • Do your competitors operate under the same constraints?
  • How much money are you losing by waiting to refine your work?
  • How much money are you making by moving on to new projects?
  • Do you actually go back and close the loop with regards to the impact of your work?
  • If so, how do you do that? How regularly? How rigorously?
  • If so, what is your batting average? What are some good / bad product decisions you’ve made in the last two years?
  • How could you speed up feedback loops such that it would make economic sense to let teams iterate based on feedback? Are there ways to reduce that risk?
  • How could you take small steps to change the organization’s focus from output to outcomes? What’s working? Where?] Is this impossible?