Ryan Dawson

Software developer at seldon.io

Why Big IT Unification Projects Famously Fail

Many people can tell you about a big IT project that was meant to save a lot of money and instead wasted a vast budget. There are some very famous examples out there — in the UK there’s the National Health Service systems unification project, in Canada there’s the government payroll unification project (Phoenix) and in the US the LA Unified School District project. Failed government projects run by private contractors get the most public attention (and costs in the billions) but there are also examples at private companies.
So what is it about these grand cost-saving IT unification projects that makes them so prone to go wrong so spectacularly?

An Example

Let’s consider a simplified example. Imagine that a consulting firm approaches the manager of a vineyard and says:
“We’ve looked at all the types of vines you’re buying and the different ways you’re planting and tending them. we can save you X by rationalising the processes, using a smaller number of vine types and bringing it all under a computer system that will tell workers when to do what.”
Anyone who knows wine is likely to be sceptical right away. Different parts of the vineyard have different growing conditions and different types of grape are therefore better suited to them. Unless the vineyard is very uniform you can’t just use the same grape variety everywhere and expect good results across the board. The consulting firm’s suggestion is therefore likely to be rejected and never get off the ground.
Now imagine that instead of approaching the vineyard manager, the consulting firm approaches an investment firm with a controlling stake in the vineyard. If the investment firm aren’t knowledgeable about wine then they might not see the nuances and might instead focus on the money to be ‘saved’ by the unification project, which for them would translate into profits and share value.
If this doesn’t seem plausible to you then substitute the vineyard for a farm. The consulting firm might look at the types of apple being grown and propose to ‘rationalise’ and bring under a single system for planting, growing and picking. The farmer would likely point out that different apples suit different conditions and have different growing seasons. But if the farm is run as an agri-business with multiple levels of management then it’s plausible that the grand unification project pitch might get a more receptive audience as it is pitched higher and higher in the management chain. The more complex the organisation is, the more levels it will have and the more difficult it will be to command a detailed picture of how it operates.
The key point here is that it takes skill and effort to see differences — it can require specialists to see all the nuances. Non-specialists are more likely to be taken in by a picture that suggests various grape or apple types are all basically the same. This applies for apples and grapes or documents and record-keeping processes and ways of doing payroll. If there are non-specialist managers who are incentivised to find cost savings, then they will be all the more likely to see all the grapes as basically the same as we humans have a tendency to see what we want to see. And this tendency to see what we want to see can get amplified by a tendency to want to report up the management chain what we think our superiors want to hear.

Optimistic Simplification

The basic driver to focus on is our tendency to assimilate different cases and treat them as ‘basically the same’. Sometimes grand unification projects can succeed because the different cases really can be assimilated or ‘rationalised’. Sometimes differences can be assimilated, sometimes they cannot and often they can be assimilated but only by accepting some trade-offs and costs. Often it is hard to determine in advance whether rationalisation will succeed. There is a tendency in large organisations for non-specialists to have most influence over this kind of decision-making. Often senior managers who aren’t afforded much time to pick through the nuances. Starting from a position with too much wishful thinking will mean the trade-offs and costs involved will effectively be unplanned.
This logic of optimistic simplification can lead to projects that are effectively doomed from the beginning. What makes it even worse is that optimistic reporting up the management chain and a desire to realise the promised benefits can mean that good money continues to be thrown after bad, even after the project has started to hit the inevitable problems. This can create situations where different sub-leads compete to report more optimistically and expand their own mini-empires within the project. Even when it becomes recognised that there are serious problems, a culture of excessive optimism can lead to chasing false saviours and desperate rounds of restructuring. Then good money is thrown after bad and costs spiral.

Complexity is Part of Life

There are of course other factors in play in the failure of big IT unification projects. The role of private consulting firms as having a vested interest in prolonging the project can get attention. But note that the core simplification logic can apply when the pitch and delivery is in-house. Government management chains can be a factor but the temptation towards optimistic simplification is not specific to government. The temptation is deep in us and our cultures, especially parts of how we run large organisations.
Big IT projects in general have a significant failure rate and can fail in various ways. Some of the simplification logic discussed can operate on projects that don’t specifically aim at ‘unification’ or ‘rationalisation’ — there is a spectrum. Big unification projects in particular are a special case with a special danger due to their size, complexity and apparent promise of such huge cost savings. Too much wishful thinking in chasing cost savings can sometimes lead to enormous calamity.



More by Ryan Dawson

Topics of interest