Agile: Revolution and Counter-Revolution

From Agile Manifesto to Communist Manifesto and back again

It is the season of the 100th anniversary of the Russian Revolution of 1917. So let’s change subjects and talk about agile software development.

Tale of Two Agiles

It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair, we had everything before us, we had nothing before us, we were all going direct to Heaven, we were all going direct the other way — in short, the period was so far like the present period, that some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only.

— Charles Dickens

It was the best of methodologies, it was the worst of methodologies. There’s a very perplexing disparity in the accounts, in internet discussions, of different peoples’ experiences with agile/scrum software development practices at different companies. There seem to be two camps, with diametrically opposed experiences.

On the one hand, some people praise the easy daily stand-up meetings, the loose structure, and the improvised “agile” nature of the work. On the other hand, there are the people who bemoan the excessive meetings, insufficient planning, and the obsessive redundant bookkeeping.

People are aware there is a disagreement, but are quick to dismiss it with a wave of the hand, saying something about “different companies are different.” But this is a missed opportunity to understand that there are identifiable underlying reasons for these differences.

While each company is different, it turns out that there is a way to categorize agile teams into precisely two categories, and you will find that these two categories provide a reasonably accurate prediction of whether the engineers will love or hate agile under that system.

To understand the simple distinction that separates the Two Agiles, let’s change subjects again and discuss the Russian Revolution.

The Russian Revolution

With little fanfare, we’ve approached the centenary of the October Revolution, which occurred on October 25th*, 1917 (*but they were using the wrong calendar.) It was a hallmark event in the overarching Russian Revolution of 1917. In honor of this historic event, I have read up on the Russian Revolution. I highly recommend checking it out, even if only on Wikipedia. In any case, I wanted to share some lesser known details I found interesting. Disclaimer: Be kind, I’m more book reviewer than historian.

“Lenin Proclaims Soviet Power,” Vladimir Serov

The Russian Revolution was an overwhelmingly popular uprising driven by the discontent of the masses. Some of the biggest factors were a continuation of the 1905 Revolution. Other factors were introduced or amplified by World War I. The discontent came from all angles — peasants, factory workers, and soldiers, whom together forged a mass majority. A lot of their shared complaints were very similar to the sentiments of other revolutions worldwide, such as the French Revolution or even the American Revolution, which both happened over 100 years prior. These were enlightenment ideals, overtaking the country on a slight travel delay. Namely, the masses of society had developed the feeling that the ruling monarchy didn’t really do anything useful, but they made all the rules, took all the stuff, and occasionally sent people off to war. It was just a useless class of entitled nobility — Enlightenment/revolution-era attitudes are probably where we get the negative connotation of “entitlement,” (implying that the “title” though claimed is undeserved.) The autocracy of Tsar Nicholas was viewed as an anachronistic bureaucracy.

So perhaps the biggest question about the Russian Revolution the following: at what point did it transform from a popular uprising to the eventual autocratic USSR we know. Was it Stalin? Lenin? The ghost of Marx? Something else?

When the masses decided they’d had enough and overthrew the autocracy, Vladimir Lenin had already mastered a form of oratory that allowed him to talk his way to the front of the movement. It is common, at least in America, for people without a complete view of history to tend to view Lenin as just some other demagogue, an earlier form of Stalin. This is not accurate. Lenin was, in earnest, an ideological Marxist, so his critiques of the Tsar and his slogans of empowerment and democracy were both genuine and consonant with the popular sentiments. Lenin’s Bolsheviks were not the most popular political party, they were close enough — most of the popular parties were left-wing socialist/communist/Marxist groups all saying the same types of things. So Lenin’s group just literally walked in the door and took the government and everybody saw it as an improvement.

The mystery of Lenin is that Marxism is little besides critique — it will tell you what is wrong, but gives no concrete prescription about “what is to be done” aside from slogans and abstractions. Russia was dragged into a civil war in which several largely incompatible groups (capitalists, monarchists, etc.) fought with backing of several European nations to defeat the Bolsheviks and either restore monarchy or some other new system to power. This led to all the Jacobin rhetoric about “protecting the revolution” and so on — war propaganda. The Bolsheviks had to learn that having some sort of stable power preceded all their theory about economic systems. Lenin died before the country ever stabilized into any form you could fairly blame (or congratulate) him for. You can’t call him a demagogue or otherwise because it is doubtful he had any concrete plans about how he could govern in a more stable climate over the long term. He made short term plans and long term slogans. It would be more fair to say he had no idea what he was doing.

I wanted to distinguish Lenin from Stalin so I can pinpoint where I think the USSR really became the big bad bureaucracy we know it as. The Bolsheviks sent everybody off to fight off the rest of Europe in the Russian Civil War and meanwhile looked around for people to help them run this new system of government they were supposedly making. The job requirements were 1) literacy 2) that’s it. They didn’t find many job candidates, so they ended up hiring all the Tsar’s old bureaucrats. The same guys! These were the guys they just got done revolting, and now they had to hire them back for a salary (freshly printed.)

So Lenin’s story arc is whatever you want to make of it, but Stalin’s was another species entirely. Under Stalin is when they created what we remember. Stalin was the guy who buoyed to the top of the bureaucracy by appealing to the other bureaucrats instead of trying to appeal to the masses. But that bureaucracy was built on the same foundation that was around from the Tsar’s times, with some of the exact same people. The saying begins, “communism works on paper…” but I don’t believe there was ever any paper. The Bolsheviks had no idea what they were doing, so they just settled right in to the system that was already in place, which was an autocratic bureaucracy.

One of the peculiar traits we know about Communism in the USSR (and elsewhere) is they wouldn’t shut up about it. Not just communism, but The Revolution. You could have a communist dictatorship 100 years old and they’ll still be saying “protect the revolution!” This doesn’t happen in normal circumstances. Nobody’s still saying any slogan about a revolution in France, or the USA. They stopped saying those things when they stopped fighting. Dictatorships and bureaucracies keep saying those things forever, because they never knew or cared what they really meant — they just picked up the parlance of the times and focused on advancing their careers. They can’t shut up about The Revolution because that’s the basis of their job. It’s all lip service.

The counter-revolution took place not on the battlefields of the civil war, but in the Kremlin within the party itself.

The Agile Revolution

That brings me back to the other revolution. The Agile Revolution.

Back in the days of yore, software developers allegedly used a Waterfall methodology, where they tried to do “Big Design Up-Front.” They would try to design an entire system before writing a single line of code. They may even write comprehensive documentation before coding. They would use UML diagrams and draw out every Java class they were going to make with some tool, before even typing any of them out. The details would all be built into the contract with the customer. All the work would be tracked and micromanaged with various tools. The whole endeavor would fall under some kind of Project Manager, as if it was some kind of huge NASA contract to build a space ship. They tried to apply the logic of an assembly line to software, because management science has never academically matured since the days where it was all about creating literal assembly lines.

But most types of software development weren’t like assembly lines, or even any type of engineering. Software development, especially on the nascent Web, was a trial-and-error process. So some of the people, especially people with experience in contract-oriented Web work, got fed up and said “we’re not doing this anymore.” Seventeen developers got together in Utah and put their new principles into the Manifesto for Agile Software Development, or Agile Manifesto. The four main tenets of agile are:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

This type of agile is still in use today. And people love it. It allows people to create software in the way that is most logical, because it doesn’t dictate a certain ritualistic set of steps you have to take (or not take!) If you want to just go write some code, write it. If you change your mind, change the code. On the other hand if you do need a big spec for something, do it. You pick the rules that make it easier and faster to get results; eliminate pain.

But this Agile put a lot of middle managers out of work, and they weren’t going to take it. Middle managers, like Tsarist bureaucrats, have a certain sense of entitlement — the feeling that they deserve a rank in life above the rest of the riff-raff. So they decided to become the leaders of Agile.

When these Project Managers come onto a project, they are trained to open with an evangelical presentation. On the first slide, they will show you the four tenets of agile I listed above. On the next slide, they will tell you about all the new processes and tools you’re going to use, and all the new planning meetings you’re going to attend, how you’ll plan all the work out as much as 3 months in advance, and pressure you to follow it as closely as possible. The hypocrisy is lost on them.

Now, these middle managers have invented SAFe, with a longer cycle, punctuated by quarterly planning. This is just Waterfall, except it has all the expense and inflexibility yet still forbids producing anything of value, like a spec or blueprint. You just get a bunch of Trello cards that you’ll have to change later — of course you’ll have to change them later, that’s why we got rid of waterfall. But then why are we trying to plan everything in advance?

So in this version of Agile, it’s got the same assembly line management as the old system, and it’s run by the same people who were originally revolted against. It’s the same guys! The old system, chanting the slogans of the new system. The Scrum Master is now a highly paid position. You report to nobody and control everybody. The job requirements are 1) literacy and 2) that’s it. This is the agile counter-revolution.

One of the peculiar traits about this form of Agile is that they won’t shut up about it. Not just Agile, but Scrum and SAFe, and all the same slogans of the Agile Revolution. Nobody wants to do anything resembling Waterfall and they’re still trying to slay that straw man. This doesn’t happen elsewhere in software. Nobody is shouting to kill Flash. We stopped slaying it when it died 5 years ago. But bureaucrats will keep shouting their slogans forever, because they never knew or cared what they really meant — they just picked up the parlance of the times and focused on advancing their careers. They can’t stop shouting about agile, because agile is the justification of their job. It’s all lip service.

Conclusion

Agile intentionally termed itself a “revolution” and used terms like “manifesto” to conjure the image of shedding a useless caste of society. They could have termed it the “Agile Enlightenment,” but they didn’t. But just like a political revolution, the same old guys come back again and again, sometimes wearing the mask of the revolution itself.

You’ll know the agile counter-revolution by who counts the Trello cards at the end of the day. If a developer runs the stand up, you’re probably in the True Agile. If your meetings are run by some weird guy who isn’t an engineer and doesn’t have any other stake in the project other than running scrum, run away. Bureaucrats want all of the power and status, but none of the responsibility. They hide under a different organizational structure. It’s not agile, it’s not scrum, it’s just bloat; it will kill your company and you will hate it.

Like the Bolsheviks, this version of Agile is only able to critique, never posit something new. They want to be leaders, but they explicitly don’t know what it is they’re trying to build — they just want a high station while it’s happening. They’ll schmooze, talk the talk, consolidate power, and purge all who oppose their agenda.

Vive la révolution!