Test Debt – What is it and Why it Matters

Test debt is one of thirteen recognized types of technical debt. It refers to the extra time testers need to allocate for completing testing activities that should have already been done. When test debt accumulates, testers can feel discouraged and overwhelmed, which causes them to get behind with their regular responsibilities.

Technical debt is created when a development team needs to apply a code change that will result in a quicker deployment, with the understanding that developers will have to go back at some point and create a more permanent solution. Test debt happens similarly when a tester passes a feature but fails to complete the related activities. As we know, testing is much more than just executing test cases; we should also be completing test activities like documentation, regressions, or automation.

For example, say you are one of two testers on a team. The other tester has gotten ill and cannot work the rest of the week. Trying to keep things on schedule, you test issues quickly, knowing that you will need to perform regression testing later. Or perhaps automated tests will need to be updated, but instead, you move on to the next item to test, knowing you will have to update the automated tests at a later date.

Author and technology consultant Martin Fowler created a grid for decision-making known as the Technical Debt Quadrant. Widely used, this concept shows that technical debt happens for four reasons. Using his principle as inspiration, decision-making options can be expressed as:-

• Careless and Intentional
• Considered and Intentional
• Careless and Unintentional
• Considered and Unintentional

If you expect to take on test debt, the Technical Debt Quadrant can help with categorization to help you make wise choices.

The Tech Debt Quadrant can also benefit testers by aiding prioritization and decision-making. It can also be helpful in retrospectives as a way to learn from previous mistakes. Here is an example of what a test debt quadrant could look like:

How is Test Debt Acquired?

Lack of resources and capacity

A lack of, or a sudden change in, resources is a common cause of test debt. For example, if the ratio of testers to developers is unbalanced, testers can become overwhelmed with the amount of work placed on them. In addition, layoffs can cause test debt because the remaining testers will likely be expected to maintain the team’s testing capacity despite the removal of resources.

Time constraints

No one wants to be the cause of a release or feature being delayed, especially if it’s because they need more time to complete their work. Testers are often bound by time limits within a sprint or iteration, which can sometimes be unrealistic.

Automation

Test automation has the potential to cause test debt because maintaining it can be challenging. For example, when user interface (UI) elements are changed, tests calling the earlier elements will break. Testers responsible for the automation repository must evaluate what broke and then update the test. As the test cases increase, so will the time needed to maintain the automated code. Another way automation can cause test debt is when analyzing the results. The code may run and test automatically, but it also takes time for someone to analyze the results of the tests.

Scope Creep

When requirements are vague, ambiguous, or prone to change, we often experience a case of scope creep. This leads to more development time and more testing time. Testers often feel the brunt of this because more work is expected but during the same amount of time.

Poor planning

Appropriate planning could prevent many of these issues. Technical finessing and sprint planning should include establishing the amount of time required for testing the feature as well as time needed for all testing activities.

Negative Effects of Test Debt

Test debt isn’t just a nuisance for testers; it can potentially cause many issues for the product and stakeholders. Below are some possible outcomes to consider when test debt is being created.

• System Stability – When regression testing is ignored or performed poorly, a usually stable system can experience failures. A rushed fix can backfire if it hasn’t been adequately evaluated.
• Reputation – A system’s credibility can be lost once issues arise. It can damage the product’s reputation, which can cause users to move to competitor products. By extension, the company’s reputation can also be negatively affected, leading users to avoid all their other products.
• Frustration – When test debt has accumulated, but there are no remediation plans, teams can feel frustrated. When this occurs, talented resources with excellent product knowledge can seek employment elsewhere. This is one of the most undesirable consequences of test debt.
• Financial – When systems fail, reputations are damaged, and talented resources are lost, which can cause a negative economic impact.

Tips for Avoiding or Managing Test Debt

Test debt can often be avoided by following proper processes. However, when test debt is inevitable, there are ways to manage it. Managing test debt means that, just like the financial debt analogy, you create a plan to pay it off by scheduling it into your time budget.

• Test planning – As mentioned above, proper test planning can alleviate a lot of test debt. When planning, allocate the time needed for all test activities, such as design, execution, documentation, regressions, automation creation, automation maintenance, and test closure activities. Communicate the plan so the entire team understands your responsibilities and values your time.
• Test reviews – Some companies require a test review ahead of executing tests. A test review is similar to a code review. Another tester on your team reviews the test cases you have created, and they look for any other relevant cases to add or further testing that needs doing.
• Innovation – In a scaled agile framework (SAFe), iterations end with an innovation week. Other agile methods include a tech debt sprint or something similar. This is an opportunity to prioritize and catch up on test debt.
• Create tasks – Create your test debt as tasks or another item in the backlog. Prioritize the debt and include time estimates while noting any possible dependencies. Doing this solidifies the concept of test debt for yourself and your team. It also offers the potential to make management, leads, or scrum masters more aware of test debt and the reality it beholds.
• Leave room in each sprint – Reserve enough room in each sprint for tackling test debt. Prepare ahead of sprint planning by knowing your capacity, then bring test debt tasks into the sprint as you are able.
• The Boy Scout rule – Leave test cases in a better state than when you found them. If you are perusing through some test cases, update them as needed while you are there. By making these updates in advance, you will enjoy the benefits of time saved in the future.

Conclusion

Test debt is a genuine concern with the potential to create serious issues. Sometimes the best choice for managing a project under certain circumstances is to create test debt to resolve later. Other times, test debt accumulates to the point that it can become overwhelming. By actively scheduling tasks to reduce test debt, teams can be more aware of their decisions, and the debt can be prioritized.