Landmines in Software Development

It’s easy to underestimate the complexity of software development. One of the challenges developers and project managers face every day is managing the Unknown. What is the Unknown, and what does it mean to manage it?

Unknowns are nothing new. Common sense suggests that nothing in life is certain. People are naturally empathetic when plans fail due to catastrophic events. However when plans fail for reasons not understood, it’s up to the planner to manage those expectations and create a plan B. For example, manufacturers that anticipate defective products set up refund policies so that customers can get their money back.

In software development, the Unknown is the difference between the developer’s current working knowledge, and the actual knowledge needed in order to make a feature work. In a world of infinite knowledge, developers would be able to describe precisely how a feature will be implemented and how long it will take to build. However, the reality is that some features will be less understood than others.

There are at least three factors that determine how quickly a feature can be built:

• How long does it take to type the code?
• How long does it take to solve the problem and create the right algorithm?
• How long does it take to find the information needed to satisfy the missing variables in the equation?

You might think that these factors are less of an issue when your team is made of rockstar developers. But the reality is that you won’t get 10x output just because you drop a rockstar team onto a project. There are many reasons why development might lag in one project versus another even with the same team:

• How many lines of code does it take to get something done? For example, developers write in C++ instead of assembly because the higher level of abstraction allows them to solve more complex problems with fewer lines of code. How modular is the codebase? When there are fewer modules, there is less opportunity to reuse code, which means code has to be retyped every time a new feature is built.
• How easy is the framework to understand? How many levels of inheritance or nested function calls, or lines of imperative code does a developer have to wrap their head around in order to know what to write next? Are there unit tests? When tests are lacking, developers have to mentally keep track of and retest every line of code they touch so that they don’t break something accidentally.
• How complete is the project documentation? How many files does a developer have to scan through in order to find the signature of a method or REST endpoint? How clear are the product specs? Have the UX mockups been created? Collecting and processing information takes time. Sure, perhaps it’s faster to write run-on sentences in the subject line of an email, but poor writing increases development time because developers have to work harder to understand what they’re supposed to build.

Developers know these problems like the back of their hand. They’re not surprised when they have to spend hours trying to figure out the correct parameters to pass in order to connect to a REST endpoint. This is why they hate giving estimates. When someone asks how long something will take to build, they’re making an intuitive guess based on past experience and their current knowledge of the situation.

Fundamentally, estimates of this kind are intuitions, and therefore unreliable.

It’s important to recognize that estimates of this kind are intuitions. They’re not data-driven. There are no hard facts backing them up. There are no probability numbers defining the accuracy of these estimates. They are just guesses.

Management that doesn’t understand the nature of these estimates, naturally assumes that they’re concrete. Intuitively, they think that spending more time calculating estimates leads to higher accuracy. To some degree, this is true. However, we’re still dealing with the Unknown. Fundamentally, we don’t know the full answer yet, and there’s an unknown possibility that we’ll hit a landmine in the process.

This is what typical software development looks like:

In this illustration, the developer starts writing code. Everything seems to be going as planned. But all of sudden, he/she runs into a landmine and the rest of the project is delayed.

What is a landmine? Here are some examples:

• A spec is written to allow users to upload a file for data processing. Except the spec writers didn’t realize that the file might be large in size, and therefore require a more sophisticated UX that can report progress updates as the user waits for the operation to finish. New UX mockups need to be designed and additional front-end code needs to be built to display the progress updates.
• The development team plans to use a certain API to hook things together, but when writing the code, they realize that the API is leaky and can’t do the things they thought it could do. Now they have to search for another solution which may or may not exist. If there isn’t an easy workaround, they’ll have to build the solution themselves.
• A spec is written to integrate feature X with feature Y. Except, not everything from feature X is compatible with feature Y due to an implementation decision that was made earlier on because of an assumption that X would never need to integrate with Y. These incompatibilities aren’t discovered until the development team actually starts writing code. Now the team has to spend additional time finding workarounds. If it’s serious enough, the integration spec needs to be rewritten.
• When developing a new feature, a developer realizes that part of the implementation already exists in another feature that was built by a different developer. Even though it’ll be faster to copy and paste the code, this isn’t a scalable way to build software. Eventually, there will be multiple copies of the code to consolidate and retest. Also, if there are any bugs, other developers will have to know to search for the other copies and make the same fix. So being a good developer, he/she decides to reuse the code from the other feature. However, it’ll take some refactoring first to make the existing code reusable. This adds a few hours to the original estimate. Except everyone does this, and now the overall estimate of the project is completely off.

Essentially, a landmine is an Unknown that results in an unmet expectation (typically a missed deadline) that has to be managed. Developers can help absorb a landmine by working longer hours or by making shortcuts in the implementation (shortcuts that create technical debt and will take more time to fix later). The product team can help prevent a landmine from affecting the launch of the product by pushing lower priority features to later sprints. But when developers are burned out, and the product team can’t defer any more features, the business has ultimately promised more than it can deliver.

Landmines also aren’t just problems that need to be addressed at a later point in time. Their damage often grows exponentially the longer they are put off. They’re basically time bombs waiting to bring down the entire project. Ignoring them just isn’t an option.