Agile Methodology’s Promise Seems To Fall Short in the Face of Reality

I’ve been on a handful of projects that have been meant to follow the Agile process off and on over the last 5 or 6 years. This has been with varying teams with different skill sets and levels of adoptions. The methodology has been applied with great vigor in my current project, with all the Agile methodology trappings, including:

• Daily standup meetings
• Backlog
• Backlog grooming
• Story points
• Burn down charts
• Scrum master
• Retrospectives
• And other blah blah blah’s

I think the thing I love most about Agile is that it recognizes Reality (capital R intended). Designing real world solutions — even very simple solutions — for real human beings is wonderfully complex for an endless number of reasons.

http://agilemanifesto.org/history.html

The Agile methodology appeals to people like me because it addresses this reality and provides a great process to manage this crazy complexity. It effectively responds to traditional alternatives that “impose irrational demands through the imposition of corporate power structures.” The thing about an irrational demand, of course, is that you simply can’t meet it, except maybe by chance.

But for all the hope I pour into the Agile methodology, I find myself frequently disappointed. Here are few reasons:

• Sprint planning sessions are simply too vague a procedure for many developers. We can never fully align on the meaning of “story point” and in many cases don’t even try. I’m personally OK with the imprecise nature and real world implication of a “story point.” Many developers, however, are not, maybe half. I’ve seen some folks more or less check out entirely because they don’t get it or like it and withdraw their intellect from the process.
• As a corollary to the above, this idea that “we plan, but recognize the limits of planning in a turbuluent environment” also kind of falls down for that class of developer who, for whatever reason, still believes in their heart of hearts that things can be neatly planned.
• The “work off the prioritized backlog until you run out of budget” thing is kind of a lie and everyone that’s been seen this movie before knows it. The real rule is “work off the prioritized backlog, but finish the whole backlog in the same budget and time as if this were a fixed project.” This means that the early sprints are relatively care free and focused on building product, infrastructure and full of high minded ideas. Intermediate sprints become a little less care free and final sprints tend toward anxiety-prone late nights, weekends and even outright panic. The final “stabilization sprint” isn’t used to stabilize things but instead to shoe horn in whatever final “must have” backlog items are still outstanding.
• Stand-up meetings are almost never conducted with everyone standing up.

The last point seems small but it actually gets to the point of this much-longer-than-expected post :).

When “Agile” projects start up, someone usually lays out the ground rules and the most easy of them all is the daily scrum meeting. They are time boxed to fifiteen minutes, team members literally stand the whole time speak to only three topics: 1) what did I do yesterday, 2) what do I hope to accomplish today and 3) am I blocked?

This usually starts off OK, but at some point, someone starts sitting and then someone else does and next thing you know, it’s not “stand up meeting” anymore, it’s a “sit-down meeting.” And here is the first little crack in the process that begins the unravelling of the discipline required to really follow the Agile methodology. Next thing, people don’t just speak to their three topics. They get interrupted. Someone responds to a blocker with a suggestion and then someone else gets involved. And next thing you know, your standup meeting kind of sort of resembles a “proper” standup meeting, but is really just another random team meeting.

Once we stop standing, we’ve discarded one of the most simple rules. It becomes easy to discard others.

There’s a similar challenge for those more literally minded developers. I’m currently entering the 7th sprint in my current project and some of my dev colleagues continue to hold up story point estimates as a kind of shield for why a user story is taking longer or less than he thinks it should have been. Or later, during task breakdown when we finally get to man hour estimates, they’ll complain that two stories with the same story points don’t equal each other’s hour estimates. We’re going into a scrum planning session this week and I just know that for the 7th session in a row, people are still going to try and translate story points into man hours and we’re going to have the same (by now) tired discussion that story points don’t equate to dev hours.

Despite all this, I still a believe that the Agile methodology is the “right” way to do things. In my heart, I also think it’s a compromise with the other “right” way to do things, which is just to do waterfall correctly :). If we can just spend the right time, with the right people to get requirements documented “just right” then we should be able to create good products. I’ve cut myself loose from waterfall because as I’ve been saying, Agile is more grounded in Reality and Reality doesn’t care what you believe in your heart.

But now I’m also losing confidence in Agile as a methodology because it seems impossible to follow it, as lightweight and sensible as it is.

My politics lean very left and a bunch of years ago (Jan 2009), I read a blog posting by Heather “Digby” Parton where she used the phrase, “Conservatism Can Never Fail, It Can Only Be Failed.” It really resonated with me and I find myself applying that phrase to non-political contexts. For instance, “Properly Implemented Waterfall Can Never Fail, Poeple Can Only Fail To Follow It.”

I’m a hairsbreadth away from believing the “The Agile Methodology Process Can Never Fail, People Can Only Fail To Follow It.” If only we’d stand up during stand up meetings! If only the scrum master would concentrate on unblocking blockers and if everyone could align on the meaning of a story point! If only everyone could be at peace with the inherent problem of *needing* to estimate even when you’re missing important information. If only this! If only that!

It’s a disappointing thing because for me, that basically means that at a macro level, nothing is really left. Success or failure is a lot more random than I want it to be on my projects and the only way to move the dial in your favor is through some kind of Hero model, dumb luck and creative definitions of “success.” Heroic project managers. Heroic scrum masters. Heroic architects and developers. Back room deals where business compromises are made by skillful business people. That sort of thing.

I’m still very invested in the Agile way because I do think it’s our best hope :). Maybe it just takes more practice with a persistent team. We’ll see.

</end>