Chapter 8: Technical Debt

This chapter discusses the concept of technical debt, which encompasses naive debt, unavoidable debt, and deliberate debt. It also examines some common causes of technical debt and the consequences of accruing high levels of debt. It concludes with how to apply three activities associated with technical debt: managing the accrual of technical debt, making technical debt visible, and servicing technical debt.

Technical Debt Overview

Technical debt is a metaphor that refers to both the shortcuts teams purposefully take and also to the many bad things that plague software systems, including unfit design, defects, insufficient test coverage, excessive manual testing, poor integration and release management, lack of platform experience, and many more. I refer to the latter kind of debt as naive technical debt, because it is often generated accidentally or carelessly.

Unavoidable technical debt, by contrast, is by its very nature unpredictable and unpreventable. It is caused by our inability to perfectly predict the future. This kind of debt is mitigated by frequent feedback loops and pivots based on validated learning.

Strategic technical debt is a tool that can be used to help organizations better quantify and leverage the economics of important, often time-sensitive decisions. For example, a company might choose to deliberately take shortcuts to hit an important short-term goal.

No matter how or why the debt is accrued, making it visible raises awareness of its costs and its consequences.

Consequences of Technical Debt

As the level of technical debt rises, so do its consequences. Some of the more notable problems include:

• Unpredictable Tipping Point. Once technical debt reaches a certain critical mass, even small changes become unmanageable or chaotic.
• Increased Time to Delivery. The greater the debt today, the slower the velocity tomorrow.
• Significant Number of Defects. Products with significant technical debt become more complex, making it harder to do things correctly and eating into the time available to produce value-added features.
• Rising Development and Support Costs. In the presence of high debt, even small changes become very expensive.
• Product Atrophy. As the cost to change or fix a product rises, the product stagnates. As a result, the product begins to atrophy and eventually ceases to be a viable option for most customers.
• Decreased Predictability. If a product has a high level of technical debt, making any sort of prediction about how long a new feature might take becomes nearly impossible.
• Underperformance. As technical debt increases, people's expectations about the product's potential decreases. These lowered expectations eventually propagate throughout the value chain.
• Universal Frustration. At a certain level of technical debt, everyone in the value chain becomes frustrated. People burn out or jump ship, abandoning the product to its fate.
• Decreased Customer Satisfaction. As customer frustration increases, customer satisfaction decreases.

Causes of Technical Debt

Both strategic and naive technical debt often are driven by business pressure to meet an important looming deadline. When a team's actual velocity trends slower than its projected velocity, the company has to make a business decision. Do they cut scope to meet the desired release date or do they add more time to the schedule to accomodate the desired scope?

In many cases, the business avoids this decision and instead asks the team to simply increase its velocity. In most cases, the team can only work at an accelerated pace by cutting back on quality or testing. These teams then begin to accrue technical debt.

A prevalent myth is that testing is additional overhead, and by reducing it, we can accelerate velocity. The reality is that reducing testing increases debt and causes teams to go slower, because problems will go undetected until later, when it is more time-consuming to fix them. That's why many Scrum teams have adopted technical practices such as test-driven development (TDD), where the developer writes and automates a small unit test before writing the small piece of code that will make the test pass.

Technical Debt Must Be Managed

The above discussion illustrates why technical debt, like financial debt, must be managed. I'm not suggesting you try to achieve a debt-free status—in most cases that wouldn't be economically justified. Teams do need ways, however, to keep debt low enough that it doesn't significantly affect future development.

Minimize the Accrual of Debt

The first approach to managing the accrual of technical debt is to stop adding naive debt. This can be acheived in several ways: 

• Use Good Technical Practices
• Use a Strong Definition of Done
• Understand Technical Debt Economics

The blog post ”Managing the Accrual of Technical Debt” discusses each of these approaches in more detail.

Make Technical Debt Visible

One of the principal benefits of the technical debt metaphor is that it enables the development team and the business people to have a necessary conversation using a shared context. The problem in many organizations is that whereas the development team has at last some visibility into the product's technical debt position, the business people typically do not. To help the business people visualize the technical debt, it is helpful to track velocity over time. The decrease in velocity can be translated in financial terms. To help make technical debt more visible at the technical level, consider using a defect-tracking system, adding the debt to the product backlog, or keeping a separate technical debt backlog.

Service the Technical Debt

The last activity in managing technical debt is to service or repay the debt. This includes the following status categories: happened-upon technical debt, known technical debt, and targeted technical debt. The following principles apply when servicing debt:

• Not All Technical Debt Should Be Repaid. Debt probably shouldn't be repaid on the product nearling the end of its life, a throwaway prototype, or a product deliberately built for a short life.
• Apply the Boy Scout Rule (Service Debt When You Happen Upon It). When a development team member is working in an area of the product and happens upon technical debt, that person should clean it up, as long as the time to do it is within some reasonable threshold established by the team. Any debt that is not serviced when found should be classified as known debt and made visible.
• Repay Technical Debt Incrementally. It is far better to make many, timely increments of “debt payments” than to try to payoff big chunks of technical debt at once. One excellent way to do this is to designate some portion of each sprint towards paying off known debt.
• Repay the High-Interest Technical Debt First. Not all technical debt is of equal importance. Teams ideally should target and service the most impactful debt first.
• Repay Technical Debt While Performing Customer-Valuable Work. An excellent way to repay known or targeted debt incrementally is to make debt payments while performing customer-valuable work. That means, we commit to doing high-quality work so as not to add to the debt, we clean up debt as we find it when possible, and we specifically plan to repay targeted technical debt in the area where we'll be working.

Closing

This chapter discussed technical debt and activities to manage its accrual. The next chapter begins a discussion of the various Scrum roles.