Never-Ending IT Projects: Navigating Complexity, Change, and Endless Development

Many projects I have worked on seemed simple at first glance. Nonetheless, there is always something to be done, and there is no end in sight. Usually, it’s not a problem—as long as there is money to finance the development. Let’s take a look at why IT projects are never done.

Complexity

Things are always more complicated than they seem at first. Often, we have unexpected complexity in two places—the problem we want to solve and the solution itself. As we try to solve the problem, we learn about new aspects that make it more complicated than it looked at first. And the solution that we create often becomes more complicated than necessary. This leads to way slower progress, to the point when the requirements change at a similar rate as the code itself.

Technical surroundings

No project is an island. Any functional piece of code depends on countless things over which we have no control. If you look with perspective long enough—5, 10, or 20 years—anything can change, from operating systems, programming languages, or ways the IT economy is running. From a business perspective, plenty of efforts take many years to become profitable.

I started my career in 2008, and over the 15 years since, I’ve seen:

• iOS (2007) and Android (2008) becoming important operating systems,

• the appearance of Node.js (2009) and its rise to popularity,

• Web platforms overtaking desktop applications,

• the growing popularity of JavaScript, and

• the disappearance of Flash and Java applets.

  
  

With changes of this scope, there is no way to create a future-proof project—at some point, you (or your successors) could be forced to rewrite everything on a new stack.

Third-party APIs

On a less dramatic scale, your application is likely to use some external providers for managing payments, sending emails, or any other services. Given enough time, some of those providers will go out of business or rebuild their API and deprecate the version that you are using. Usually, there is some transition period, but it will likely be a few months rather than a few years. In a project with many competing priorities, this can feel like short notice.

Legal Requirements

Similarly, the legal requirements can change from one year to another. In the EU, we had a big change with VAT and online sales in 2021. For many online companies, that meant plenty of changes in the code. They were forced to start dealing with VAT in every EU country that they were selling to.

The company where I work deals with brick-and-mortar businesses—we were not affected by online trade regulations. Nonetheless, we had an assumption in the code that there are only three levels of VAT—for example, 0%, 7%, and 21%. This worked fine for us for a long time. It turned out to be a problem, however, when one of our customers expanded to a new country. Over there, not only were the percentages different, but there were also simply more levels. When you have to change a fundamental assumption like this, you are likely to get an avalanche of changes in your codebase.

Code dependencies

A mature JavaScript project has many essential dependencies:

• the framework it uses

• unit test library

• end-to-end solution

  
  

Creators of those tools try to smoothen the upgrade path as much as possible, but they have a trade-off between the ease of upgrade for you and how much they can change. And when the tool is discontinued, moving to a replacement is guaranteed to be a lot of work.

When I started working with AngularJS, Protractor was an impressive end-to-end tool—the best tool available for doing integration testing of angular applications and probably for any front-end application. Now, 8–9 years later, it’s been a few years since it got an update, and in August 2023, it reached its end-of-life. There are better tools available now, but moving the four hundred tests I have on Protractor would be a huge project.

Product discovery

Product discovery is a nice term that covers the fact that it takes time to find the right combination where:

• your customers have a problem,

• you can address this problem at scale with an application,

• they are willing to pay to use your solution and

• what they pay covers your costs.

  
  

You can expect this process to take time and many iterations. Some of the iterations will require implementing a feature and then seeing if it's useful to customers and whether it makes money for the company. As long as the product is used, there will be some ideas worth checking.

How to manage it

So we know that the IT projects are unlikely to be done—that there will always be some changes that require updates to the code. What can we do about it?

Plan long term

There are always deadlines to be met, but make sure that quarterly goals will not ruin the long-term future. Think about it as a marathon, not a sprint. One thing worth investing time in is code quality. Some compromise and flexibility will be needed, but think about the overall architecture, consistency, and best practices of the stack you use. Rushing too much can lead to making a code so difficult to maintain that it will become impossible to make the simplest changes.

Document what you are writing. For a start, maintain a README file with instructions on how to start the project. Write documentation for classes and methods, preferably in a way that makes it easy to extract the documentation—for example, JSDoc.

Write automated tests. Unit tests slow down development when you are learning to test at the same time as you write code. Once you are proficient with testing, developing tests along with your code can be faster than the code alone. When you write code and tests together, you have a quick way to recheck all the edge cases you want to cover.

Keep the team around.

Even with the best effort at documentation, there is a lot of knowledge that is stored only in the minds of your team members. What decisions were made four years ago and why? How were similar bugs fixed the last time? What code is responsible for what? Every time a team loses someone, part of this knowledge is irreversibly lost. It’s a shame the industry got to the point where changing jobs often is the best advice for developers. If you can influence that, it would make sense to make sure the devs are happy with the job and the conditions and stay as long as possible.

Also published here.