Technical debt

Technical debt2018-07-16T22:21:55+00:00

What is technical debt?

One of the most difficult aspects of systems development is technical debt. In their CRASH Report on Application Software Health – 2011/12, the research organisation CAST Software defined it usefully as:

Technical Debt represents the effort required to fix problems that remain in the code when an application is released.

The idea of technical debt was introduced in the 1992 by Ward Cunningham. He used it to describe how engineering work can be effective even though it is not efficient, elegant or of especially high quality. You promise yourself that you will rewrite the bad code, document it properly, pull together a proper set of tests, etc., etc., but somehow this never happens…

As he put it:

Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt.

Or as Grady Booch put it:

The concept of technical debt is central to understanding the forces that weigh upon systems, for it often explains where, how, and why a system is stressed. In cities, repairs on infrastructure are often delayed and incremental changes are made rather than bold ones. So it is again in software-intensive systems. Users suffer the consequences of capricious complexity, delayed improvements, and insufficient incremental change; the developers who evolve such systems suffer the slings and arrows of never being able to write quality code because they are always trying to catch up.

In short, the poor quality of the original work has created ‘technical debt’ that, like a financial debt, will have to paid for later, in many ways:

  • In the cost of fixing the original problem.
  • In the cost of managing the impact of these problems on users, customers and stakeholders.
  • In the cost of working with the unknown effects of a defective product – often in quite unexpected areas.
  • The ‘cost of delay’, including both the revenue that will not be received while delivery is delayed and the ‘opportunity cost’ of things you will not now be able to do because you will have to waste budget on fixing unnecessary problems.

And of course, keeping unplanned debt under control also makes it a lot easier to take on more debt when it’s a sound investment.

As has often been observed, out of the traditional triangle of engineering goals – fast, cheap and good – you can only have two out of the three. When you prefer ‘fast’ and ‘cheap’ to ‘good’, you’ll still have to pay for it one day – in later work, when you will have to go more slowly and incur more cost because of the mess you (or someone) created before.

How big is the problem?

All software engineers are familiar with technical debt, but quantifying it is more difficult. However, it should never be underestimated. As Cunningham also said:

Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation…

CAST found that:

Based on this definition and the analysis of 1400 applications containing 550 million lines of code submitted by 160 organizations, CRL estimate that the Technical Debt of an average-sized application of 300,000 lines of code (LOC) is $1,083,000. This represents an average Technical Debt per LOC of $3.61.

CAST also calculated debt levels for particular languages:

CAST 2012 Technical Debt By Technology

Deloitte suggested a similar figure – about $1m per business system.

Is technical debt always a bad thing?

Like other kinds of debt, technical debt isn’t always a bad thing:

  • If you are confident of being able to clear up your debt in future, it may be worthwhile taking on debt now because the return on this ‘investment’ will exceed any realistic costs (bugs, complicating other developments, additional refactoring, etc.) in future.
  • In an emergency, it’s often worth taking on technical debt for the sake of solving the problem quickly & potentially forestalling problems elsewhere.
  • One-off, highly disposable products (e.g., temporary web pages) may not be worth perfecting.
    • However, it may still be possible to come up with ways of making even one-offs cheaper, more reliable, and so on, such as templates.
  • Where it is not clear what the long-term value of the product will be – including whether it will continue at all – eliminating technical debt early risks making an expensive investment of doubtful value.Exploratory work is frequently messy, but that does not mean that it’s worth tidying up straight away.
  • As Henrik Kniberg argues, new debt is not generally a bad thing – it’s usually old mess that’s causing the real difficulties.
  • Sometimes the speed with which you deliver is more valuable than the quality of the product, especially if the user does not suffer.
    • For example, seasonal products that arrive late are of no value, no matter how high the quality.

And so on.

The problem of technical debt is not that it exists but that it’s allowed to accumulate behind your back, so to speak. Eventually it reaches the level where you’re spending more time fire-fighting technical crises and dealing with angry stakeholders than you are spending creating solutions to their problems.

But experience shows that organisations often ignore technical debt altogether, preferring delivery of full scope on time and on budget, regardless of the price they will eventually be forced to pay for it. Sometimes this is done knowingly, by permitting a few residual defects into a release. Sometimes the approach is less honest, when key stakeholders insist that bugs that were originally classified as high-severity or high-priority should be downgraded so that the release can meet its deadline.

Minimising your technical debt also makes it much easier adapt to more radical changes the industry throws at you unexpectedly – transforming a well-defined architecture, working with new languages and concepts and paradigm shifts are all a lot easier to manage if you can count on your code to do what it’s supposed to do.

The basic assumption in Agile, by contrast, is always that quality is crucial to long-term success, so you should always default to assuming that technical debt should be minimised wherever possible. If you decide otherwise, it should be for a very good reason.

Causes and effects

Symptoms of technical debt

Broadly speaking the symptoms of debt can be summarised in a small number of quite familiar concepts:

  • Anti-patterns.
  • Code ‘smell’.
  • Balancing the scope-quality-cost triangle.

It is possible to go into great detail about what counts as technical debt and what its detailed effects are – and there are many websites and publications dedicated to such detail. However, the major symptoms of technical debt are familiar to most professional developers:

  • Unreadable code.
  • Unmanageable complexity.
  • Impaired adaptability.
  • Unmaintainable systems.
  • Obsolete documentation.
  • Increasingly unrealistic plans.
  • Rapidly worsening delivery economics.
  • Progressively degraded relationships – with the business, with management, with other teams, with each other.

Diagnosing the root causes of technical debt

So where does technical debt come from? Sometimes it is the result of a ‘point’ decision – a one-off choice to not do something properly, which then comes back to haunt the user, the development team and the organisation as a whole.

Other causes are more systematic. They express shortcomings in the way work as a whole is organised. There is no shortage of such causes of technical debt – in fact almost every aspect of development, from requirements management to project control to business-IT relations is a potential source of future debt.

Here are some of the major causes of technical debt:

FactorImpact
Stakeholder pressureOften the business or executive management will force the delivery organisation to release a product before it is technically ready.

This is a problem only when there is no subsequent investment in repaying the technical debt this leads to, but unfortunately this appears to be the norm in many organisations: businesses are generally blind to the idea of technical debt, and make decisions without understanding – or caring about – the implications.

Weak storiesLong before a single line of code is written, weakly defined stories make technical debt almost unavoidable. In the absence of clear, consistent statements of what the business wants or how the users say the solution should work, developers will be working in the dark. The solutions they produce will almost certainly be loaded not just with mistakes but also with technical debt.
Lack of standardsIndustry-standard standards, frameworks and technologies are ignored and their internal counterparts are not developed or applied. As a result, solutions are of poor quality, integration is hard, and new team members take a long time to acclimatise to the systems.
Lack of process or methodBecause there is no well-defined process for carrying out work, it proceeds in an uncoordinated way. Local decisions are made without enough knowledge of the wider picture, leading to mistakes and weak and uninformed decisions that undermine work elsewhere.
Lack of collaborationIn the absence of regular communications between individuals and teams, silos are perpetuated. Knowledge and good practice are lost (or at least not shared), teams become very uneven, junior developers are not coached and the quality of work as a whole suffers.
Poor design & programming practicesEither because they are not understood or pressure for a ‘quick fix’ prevents them from being used, sound programming practices such as abstraction, modularisation, loosely coupling components, encapsulation, etc., are not used. This leads to solutions that are buggy and unadaptable to future changes in business needs.
Lack of test strategyTests are not defined properly or lack a clear reference (e.g., a specific requirement or design element), or are not documented or organised into a stable test suite. As a result, testing lacks a wider system perspective, individual tests are frequently faulty or have to be reconstructed each time testing is needed, ‘fixes’ are ineffective, short-term and risky and there is no basis for the kind of automation that would prevent both defects and debt from accumulating.
Lack of documentationAlthough Agile de-emphasises documentation, it’s still important that enough information should is documented – for example, technical background & architecture, security, data structures, security, and so on. In its absence, software development is likely to rest on misunderstandings, errors and omissions.
Parallel developmentWhere multiple teams or individuals are working on the same code base but do not have the disciplines or technology needed to maintain more than one branch of code, the final merging of changes into a single source base is extremely likely to be difficult, time-consuming and risky.
Failure to refactorIt is sometimes sensible to allow technical debt to accumulate, but never sensible to allow it to remain. Refactoring is the process of re-engineering a system to remove technical debt. If this practice is neglected – e.g., because project budgets cannot be spent on refactoring – then the cost of technical debt will be especially high.
Developer ignoranceIn the absence of adequate training or professional development, developers simply don’t understand good practice, current technology or even, at its most basic, how to write elegant code. Instead they hard-code fixes and data, fail to comment code usefully (code branches, calls, CASE statements, variable definitions, etc.), jump in and out of modules, and so on.
Weak change controlChange is good: Agile thrives on change. But uncontrolled change is very bad, as it means that different parts of the organisation will have different understandings of what is expected. This will lead not only to mistakes but to bad design and implementation decisions with long-term implications – technical debt, in fact.

Key events in accumulating technical debt

There are also key events in the delivery process where management decisions almost ensure an increase in technical debt. Sometimes this is unavoidable or a price worth paying for a larger benefit, but often it simply reflects a lack of understanding.

Weak Definition of Done

An early moment where technical debt is very likely to be incurred unnecessarily is where a weak Definition of Done is accepted – or later, when a good definition of done is weakly enforced. This is shown in the following diagram:

Robust and weak definitions of done

The problem is clear. A weak definition of done means that a delivery is permitted before all the (objectively) necessary checks are made and the deliverable is really good enough. This doesn’t affect only the user – it also means that there are likely to be problems with the deliverable that will also make the team’s life harder in future. This doesn’t just mean bugs; it can also mean significant design decisions that ‘work’ but limit the system’s flexibility, making it more expensive to change or improve it in quite ordinary ways.

Forcing the pace

Another situation where technical debt is often created is where the burn chart shows that an iteration is likely to overrun, but instead of cutting stories out of the iteration backlog, the pace is forced. The results are illustrated in these two diagrams.

Forced Pace and Technical Debt 1

In the first, the team notices from the burn chart that they are likely to miss the target release date. In Agile, the recommended solution would be to remove the lowest priority stories, hit the target date and then include the omitted stories in a later iteration. In this case, however, the decision is taken to force the pace of development so that all the stories remain in the plan and they are delivered on time. In other words, a core Agile principle has been abandoned.

Forced Pace and Technical Debt 2

But the team pays a price for this decision. In the rush to finish on time, many shortcuts will be taken, many items will be finished poorly, testing and documentation will be skimped and refactoring neglected. In the end, the team will have created more work for themselves – either by reducing the quality of the code or by needing to work harder on technical debt. By speeding up in one iteration, they will be forced to slow down later.

The Agile approach to technical debt

So how do we prevent (unwanted) technical debt? Going by the above list, the long-term answer is a radical overhaul of how we do systems engineering. In fact almost all the forms of technical debt described here arise from ignoring the basic Agile disciplines.

How then does Agile approach technical debt?

The Agile approach

In Agile, the approach to technical debt is simple and direct:

  • Avoid technical debt by working to high standards, for example by including work on technical debt in every release or even iteration.
  • If you must incur technical debt, do it in a planned and organised manner, including a clear understanding of what the impacts are going to be and how you are going to pay the debt back. For example, technical debt becomes more visible if the developers work closely with architects and designers of related systems and as well as with their own users.
  • When you can’t avoid it, repay the debt as quickly as possible by focusing on areas where you know you have sacrificed quality for cost or speed. That way you will avoid a good deal of the future ‘interest’ on the debt.

Practice steps for avoiding technical debt

There are quite a few steps teams can take to avoid or minimise technical debt. Some of them are basic Agile practices, but there are others:

  • Build repayment of technical debt into the development programme.
    • Include stories within each release to repay past technical debt.
    • Set up projects specifically to repay debt.
    • Estimate how much technical debt has been accrued that will affect the current release, and adjust stories and iterations accordingly.
  • Design to avoid or minimise debt:
    • Model architecture in advance of development.
    • Include architects in the development team.
    • Explicitly review designs for debt – both for how old debt can be repaid and how new debt can be avoided.
  • Manage technical debt actively within releases.
    • Factor technical debt remediation as a fixed percentage of release budgets and schedules.
    • Include remediating technical debt as a goal for ‘hardening’ iterations.
  • Build awareness of technical debt.
    • Train your team to understand, identify and minimise technical debt.
    • Brief senior management and business partners on the economics of technical debt.
    • Track costs and benefits of managing – and not managing –technical debt.

For more on how technical debt is managed in real teams, see here.

Key agile practices and disciplines

Are there any other ways to prevent technical debt from accumulating on a less strategic level? There certainly are – in fact many fundamental Agile practices have exactly this in mind. Here are a few:

Review these areas to see how Agile prevents technical debt from accumulating.

More…

Technical debt is a popular internet topic, although relatively few sites are really valuable. Here are a few of the best:

  • OnTechnicalDebt – an online community discussing Technical Debt.
  • On ‘code smells’:
  • The Wikipedia article on anti-patterns, which identifies a range of methodological, design, programming and other mistakes that are likely to increase technical debt.

Leave A Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Want to do more than just build systems?