Understanding Technical Debt As A Software Engineer

What is tech debt? How to manage it effectively? In this Engineer's Complete Guide to Technical Debt, you will learn metrics, statistics, and tools that help engineering teams accelerate the software development process.

Debt is a complicated subject. The world often carries negative connotations with images of student loans, medical bills, or mortgage payments often coming to mind. In reality, though, some financial debt can actually help people.

The same thing is true with technical debt.

Technical debt – also known as tech debt or code debt – is what happens when a development team speeds up the delivery of a project or functionality that will require refactoring later on. A quicker development process becomes the priority instead of high-quality code.

Like financial debt, tech debt can either harm or help your organization. In order to use it wisely, engineers and team leaders must monitor how much technical debt they acquire and learn to manage it well. This can become a difficult task for organizations, especially when they do not have a clear understanding of the advantages and disadvantages of tech debt.

That’s why we wrote up this complete guide to technical debt, to help bring clarity to a new and somewhat murky conversation.

This guide will help you answer the following questions👇

• What is technical debt?
• How do you define it?
• What synonyms are there for technical debt?How is technical debt used in Scrum and how do you handle it?
• What are the different types of technical debt?
• What is Martin Fowler’s “Technical Debt Quadrant”?
• What type of tech debt should always be avoided?
• What are some technical debt metrics?
• What are tech debt statistics?
• How can you start managing your technical debt?

Let’s get started!

What Is Technical Debt?

Veterans of the software development industry have probably heard the term “technical debt” more times than they can count. Even newbies have probably heard people use the debt metaphor before.

But even though the phrase gets thrown around a ton, many people don’t actually know what it is. Ask a team member for a definition, and you’ll likely see them stumble over their words and suddenly dart out of the conversation.

So what exactly is technical debt?

How do you define Technical Debt?

Software professionals fight over the technical debt metaphor because metaphors have a natural openness to them. Ward Cunningham, the developer of the first wiki and co-author of the Agile Manifesto, first used the tech debt metaphor simply as an illustration.

Cunningham has elaborated on the concept since first using the metaphor decades ago. In one popular video posted in 2009, he explains that...

“With borrowed money, you can do something sooner than you might otherwise... I thought borrowing money was a good idea, I thought that rushing software out the door to get some experience with it was a good idea, but that of course, you would eventually go back and as you learned things about that software you would repay that loan by refactoring the program to reflect your experience as you acquired it.”

The rushed development process means that the code base will have certain deficiencies that a programmer will have to rework or clean up later on. These deficiencies, also called “cruft”, affect the overall code quality, and although the software can still function, it cannot reach its full potential until someone fixes the deficiencies. 

Fixing the cruft, then, is like paying interest on a financial loan: it’s the price you pay for expediting the process and neglecting it can have really negative effects.

This brief history lesson provides a foundation for a new definition of Technical Debt:

Technical Debt is a metaphorical framework for thinking about the cruft that needs refactoring after a development team expedites the production of software.

What synonyms are there for technical debt?

Like most coined terms, technical debt goes by a bunch of other names that all essentially mean the same thing. When talking about technical debt, you might also see things like…

Tech debt: a simple, shorthand nickname for technical debt.

Design debt: technical debt specifically having to do with the software’s design elements.

Code debt: technical debt associated with bad code in a software system that a programmer has to refactor.

Each of these have minor differences, but they all fall under the larger category of technical debt.

How Is Technical Debt Used in Scrum/Agile and How to Handle It?

Scrum has become a popular framework for software developers to use as they seek to deliver their products in a more efficient manner. One key principle with Scrum is that things are unpredictable: customers change their minds and new needs arise often. This openness to change means that technical debt will likely occur when using the Scrum framework.

One Scrum trainer – Stefan Wolpers – talks about two different types of tech debt in Scrum.

First is the active choice to create a short term solution consisting of less-than-perfect code so that you can deliver the product more quickly. The expectation is that the development team will go back and improve the code quality after the initial release.

The other type of tech debt happens more passively as the development team discovers more information about the problem they’re trying to solve. As new needs arise, solutions that worked yesterday may not work tomorrow. The code requires adjustment and refactoring, and in this way, it contains a certain amount of debt.

Wolpers points out that the Scrum Guide does not give any specific guidance regarding technical debt. It offers no “silver bullet,” as he puts it. Despite this, he also recognizes that accruing technical debt is an inescapable part of the business, and suggests 6 ways for Scrum teams to better handle their code-based financials.

According to Wolper, Scrum teams should…

1. Prioritize transparency with technical debt. Transparency, says Wolpers, makes managing technical debt much simpler. He recommends that teams prominently display a visualization of their technical debt to keep it at as a front-of-mind priority and that they review their tech debt needs during every Sprint meeting.
2. Track tech debt. Wolpers suggests counting bugs and using more in-depth code metrics like cyclomatic complexity, code coverage, SQALE-rating, and rule violations if possible.
3. Pay back debt quickly and regularly. Like financial debt, technical debt is best managed when teams pay back their loans on a regular basis. Wolpers says that Scrum teams should consider allocating 15-20 percent of their resources to refactoring code and fixing bugs every sprint cycle.
4. Align product backlog. Your product backlog should include tasks related to paying back technical debt. Organizing your debt into what you want to tackle this sprint will help keep the debt from falling through the cracks and being forgotten about.
5. Adjust your definition of “done.” Don’t count something as finished until it meets a set standard for manageable tech debt.
6. Standardize procedures. Develop a repeatable formula for how your team handles adding experiments or new features that include the introduction of technical debt.

What Are the Different Types of Technical Debt

Technical debt can take on many different forms, and people like to argue about the differences.

At the very least, most experts agree on two different types of technical debt: intentional and unintentional. Steve McConnel gets credit for recognizing this distinction in 2007, but it is similar to Wolper’s separation of active and passive technical debt for Scrum users.

Intentional tech debt (also called deliberate or active) occurs when organizations choose to leave room for improvement in their code for the sake of reducing time-to-market.

Unintentional tech debt (also called accidental, outdated, passive, or inadvertent) happens when the code quality needs improvement after a period of time. This could be the result of poor production the first time around or simply the natural need for updates as code becomes outdated.

One academic paper published in 2014 titled “Towards an Ontology of Technical Debt” rejected the idea of only two types of tech debt. They felt that organizations would be better served if their technical tech had more specific categories, so they came up with 13 different types of technical debt, each including the specific problem within the title:

• Architecture Debt
• Build Debt
• Code Debt
• Defect Debt
• Design Debt
• Documentation Debt
• Infrastructure Debt
• People Debt
• Process Debt
• Requirement Debt
• Service Debt
• Test Automation Debt
• Test Debt

While these separations have offer specifics, the most popular way of categorizing debt comes from Martin Fowler’s Technical Debt Quadrant. Like Ward Cunningham, Martin Fowler was one of the 17 authors of Manifesto for Agile Software Development. When it comes to technical debt, however, Fowler went solo and developed what he calls the “Technical Debt Quadrant.”

What is Martin Fowler’s Technical Debt Quadrant?

In 2009, Fowler nuanced the twofold separation of intentional and unintentional technical debt popularized by Steve McConnel. He felt that people used the debt metaphor to ask the wrong questions.

Instead of trying to dig out an answer to technical questions about design flaws like “is this considered technical debt or not?”, Fowler wanted to know if the debt these software systems accrued were reckless or prudent. This distinction, coupled with the idea of “deliberate” or “inadvertent” debt created what became known as the Technical Debt Quadrant.

The quadrant creates four different types of technical debt:

• Reckless and Deliberate
• Reckless and Inadvertent
• Prudent and Deliberate
• Prudent and Inadvertent

The inadvertent and deliberate debt is the same as intentional and unintentional: deliberate debt happens as a choice in creation and inadvertent comes after the fact when teams need to make adjustments.

The prudent and reckless distinction is a bit more unique, though, and this piece gives the quadrant its value. Prudent tech debt comes from a team that knows what it’s doing while reckless debt accrues when people have just been too sloppy.

Fowler shows this distinction in the following graphic:

Can you see the difference between the prudent and the reckless?

The prudent team understands the moves they make. They’re using their tech debt intentionally.

The reckless team just treats their software system like an American Express cardholder on a shopping spree. The debt just keeps coming.

Fowler does note that the most complex part of the quadrant to explain with the debt metaphor is the Prudent-Inadvertent cell. This comes when a team knows what they’re doing and does a good job on the project but still ultimately needs some reworking later on.

All software engineering teams, he says, should expect some level of this debt no matter their expertise or experience. A little debt, when prudent, is to be expected, but that just makes it all the more valuable to keep the reckless debt down and to reduce the bad code as much as possible.

What Type of Tech Debt Should Always be Avoided?

Prudent tech debt can have a lot of benefits for software organizations, but these same organizations should keep an eye on how much reckless tech debt they accrue. The reckless is never good, but another form of technical debt exists that can harm organizations even more: bit rot.

Bit rot (also called “software entropy”) happens when software deteriorates over time to the point of creating errors or even altering its functionality and usability. Bit rot takes time to develop, but it can create a huge head for development teams.

It usually happens when developers make small, incremental changes to legacy code that they don’t fully understand. These small changes eventually create enough complexity and problems that the whole software gets affected. Some engineers may even violate NFR (Non-functional requirement) or break the code entirely. The only way to fix this type of tech debt is to refactor the whole thing.

The biggest problem here in terms of debt is that the small changes actually contribute to the total amount of debt, and most of the time, the teams don’t even know it. Using Wolper’s concept of transparency can help organizations avoid such a disaster.

Likewise, teams would benefit from fully understanding each piece of software they work on so that they do not unintentionally add code that can hinder the system. Project management teams can keep their developers accountable by making sure their development processes do not leave room for bit rot to occur.

What Are Some Technical Debt Metrics?

Knowing a lot about technical debt does not really matter unless you can measure it.

But what should you measure?

Like any good management plan, organizations need to know the best metrics in order to gain control over their technical debt.

Here are some of the best metrics to look at:

Bugs

At the very least, software developers should be counting and keeping track of their bugs. This includes both fixed and unfixed bugs.

Noting the unfixed bug allows development teams to focus on and fix them during their Agile iterations. Noting the fixed bugs helps teams measure how effective their tech debt management process is.

Code Quality

While bugs have a more direct effect on the software’s end-users, code complexity can really damage the development team and the organization as a whole.

Look for code complexity metrics like…

Cyclomatic complexityClass couplingLines of codeDepth of Inheritance

The lower each of these measures, the better.

Keeping an eye on these metrics also helps organizations know exactly which code to rework or refactor in order to reduce complexity and improve the backend side of the software.

Code Cohesion

Like code quality, focusing specifically on code cohesion will help the code from getting too complex. A high code cohesion usually means that the code is more maintainable, reusable, and robust. It also minimizes the amount of people who need to get involved in the code, which can greatly reduce complexity and decrease the chances of bit rot.

High cohesion is when you have a class that does a well defined job.

Code Ownership

More engineers usually means more cruft, and more cruft often leads to greater problems and higher levels of unintentional technical debt. That’s why code ownership is such a valuable metric: it answers the question, “who is focusing on what code?” If something breaks, who do you call?

This metric will give your project management eyes on the amount of people working on various pieces of code. Knowing this information will enable these teams to reduce the amount of people and time dedicated to these efforts. You don’t want one person owning a complete section of code - just in case they get ran over by a bus or, in a better scenario, a new job. Usually, the right level is for teams of engineers to own domains in the codebase.

Churn

Code 'churns' when it gets rewritten / replaced. It's a measure of the activity a given piece of code sees. You want to give special attention to code that sees a lot of activity because any problem in there will be exacerbated. Measuring churn, then, helps teams recognize and prioritize what parts of the code require refactoring. If engineers have to constantly address bugs around the same part of the code, that means something is going wrong over there. Watching for that churn will help organizations pinpoint those problems more quickly, allowing them to fix the problem with a permanent solution that keeps tech debt down.

Keeping track of these tech debt metrics will not eliminate all your debt, but it will help you manage it more effectively.

What Are Tech Debt Statistics?

A handful of academic studies and surveys have been conducted that shed light on the software industry’s view of the technical debt metaphor.

One industry survey from Carnegie Mellon’s Software Engineering Institute found that the majority of participants found some value in the metaphor though they had slight discrepancies of specific definitions. Even more intriguing, however, is their findings regarding what contributes to technical debt.

Respondents said that poor architecture choice was the primary source of their technical debt. Second was overly complex code, and third was closely tied between a lack of documentation and inadequate testing.

These results show that most of the respondents accrued debt unintentionally. What’s worse is that 65% of participants said that they had no process in place for managing their technical debt. This means that they accrued tech debt because of a lack of strategy and have chosen not implement a strategy to fix it!

One software developer who has over 20 years of experience working with all sorts of companies, from Fortune 500s to startups, suggests that organizations should invest in their tech debt like they would invest in paying off credit cards. He says that organizations should focus their investment in two places:

1. Their company culture
2. Their codebase

Investing in the company culture looks like getting everyone on the same page. It means putting systems in place that keep people accountable. It means knowing what people are working on.

Investing in the codebase may mean performing more in depth and more frequent quality assurance tests. These tests may even be automated. It could also mean more frequent refactoring, especially if your organization has already accrued a good deal of tech debt. At the very least, investing in your code base should include some form of code reviews to make sure that the problems get solved before they get way too out of hand.

Investing in these places are great places to start, but the most valuable thing any organization can do is manage their technical debt. Without a clear strategy for doing so, the debt will just keep rising and create more problems further down the line – like it did for those companies from the survey.

How Can You Start Managing Your Technical Debt?

Like financial debt, technical debt will only decrease if you make a plan to manage it.

Managing technical debt does not come easy. It requires frequent monitoring and diligence, but it has become a necessary part of any software company. Tech debt can easily get away from your business objectives, so managing it helps keep your development aligned with the rest of your organization.

Still, managing technical debt can feel like a burden. Project managers, product teams, and engineers are already overwhelmed enough. Isn’t that why they’re accruing debt in the first place? Won’t adding another thing to their plate only increase their stress levels? Maybe so, but organizations that want to keep their tech debt low have to make it a priority. It’s just that big of a need - especially when you look at the statistics.

By 2024, global tech debt that has not been fixed will cost companies $4 trillion. The companies who do remediate technical debt see a 50% faster delivery times to their customers.

In the last few years, new technology applications have recognized this need in software companies and have sought ways to meet it.

Now, there’s software like Stepsize that helps your organization manage its technical debt by making it easy for development teams to report debt, sort the debt reports, and identify the most important debt pieces that need addressing. All this helps software engineering teams manage their technical debt without drastically altering their regular workflow. More importantly, it empowers them to quickly produce great software while monitoring all the technical debt they build up.

Check out Stepsize! We built a SaaS product to do all that, and you can try it for free :)

Previously published at https://blog.stepsize.com/complete-guide-to-technical-debt/

(Disclaimer: The author is a Co-founder & CEO at Stepsize)