Agile Test Case Management – Keeping Your Test Cases Lean

A test case repository can easily grow too large and full of obsolete test cases if not regularly managed. A number of test cases will be executed by QA, reported as failed and then closed by the product owner, to then become an unnecessary waste of time and resources. We’ve all been there. A test case is written from a user story, then several sprints later, a new user story changes the workflow or the intended functionality and a new test case is written. If the previous test case is not updated or deleted, it can cause a round of confusion from QA to Dev to Product. To help avoid this situation, I’m going to provide some tips for efficiently managing these test cases in an agile environment. By being proactive with agile test case management within each sprint, the QA team will avoid accidentally executing obsolete test cases, and avoid filing invalid bugs, so creating a more efficient process for ensuring test coverage.

The Problem With Agile Test Case Management

One of the biggest mistakes a QA team can make is to dump all test cases into one bucket to be used for regression. I’ve seen this done a lot, because at the time it may appear there are more important issues to focus on. This not only adds obsolete test cases to the regression test suite, but significantly increases the demands placed on the management of test cases in the future.

Here is why: What often happens is a tester creates the test cases from a user story during a sprint. The test cases are valid and necessary for testing during that sprint, but at the end of the sprint the test cases will be moved into a bucket that includes all test cases from all previous sprints. The tests pile up and when it’s time to do a full regression, the team discover a massive amount of test cases, including duplicate test cases and invalid tests. Usually, there is a small window in which to execute the full regression and now the team must sift through all the tests to figure out which of those to execute. Executing the wrong test cases at this point is a productivity killer and will lengthen the time needed to complete the regression. The last thing QA wants is to delay a release because they were not properly prepared to complete regression testing.

The Solution to Managing Agile Test Cases

The solution is simple really, it just requires a level of proactivity, to prevent the team scrambling to update regression tests at the time of regression. Schedule time within each sprint to manage regression test cases. The changes for each user story will fall into one or more of the following four categories. Evaluate each user story and the tests created, then decide which suite or category or categories the tests for the user story falls under. It’s possible that a single user story will require tests to be added, updated and removed at the same time.

• Add Tests – If major functionality is introduced, the tests created for the sprint testing should be added to the regression set. This may include adding the test to a smoke or sanity test as well. Where the test is added will be determined by the significance of the functionality or feature. If it is a high level feature that a majority of user’s will use, then it should be added to both regression and smoke/sanity testing. Adding a test is usually done by default which will cause the issues I mentioned above. Be sure the test is one that should be added to regression to avoid duplicates. Which leads us to the next option; updating tests.
• Update Tests – In the event that a workflow is updated but, for example, no significant changes are made to functionality, it is not necessary to create new tests. Simply editing the tests to match the new workflow will be sufficient and in this case, the tests for this functionality in the regression suite should be pulled into the sprint and updated to match the new workflow or behaviour. If the test case author were to simply write a new test instead of updating the earlier one, then you have a duplicate/obsolete test case in regression. If this happens over and over you suddenly have a major headache on your hands when you start regression testing. Remember to update the existing test case before pulling it into the sprint test plan and that the original test should be updated at the source and then pulled into the sprint.
• Remove Tests – Some user stories simply require removing a feature that is no longer in use. This might be because a provider has been removed and content that was tested previously is no longer relevant. When it comes to updating test cases, it is just as important to remove unneeded details as it is to add newly relevant ones. If a new tester is hired, they will have no previous knowledge of the history of the product. When the tester is assigned a set of regression test cases that are obsolete, they will be entering invalid bugs. This wastes the time of the tester who enters them, the product team who reads and comments on them, and the senior tester who must communicate the reason for the bug being invalid. One invalid bug turns into an unnecessary waste of resources. It can all be avoided by removing the test case when it is no longer valid.
• Do Nothing – User stories that simply adjust an image, color scheme or text should have test cases created in sprint testing. These tests don’t belong in the regression suite. They are valid tests, and may need to be executed each time the software is deployed to a new environment, but after that they should be set aside. Elements that change frequently and are highly unlikely to be touched unless another user story is created, are not useful in regression testing or smoke/sanity testing. In this case, after the sprint is completed and the change is in production, leave these tests in the sprint folder for historical purposes.

Conclusion

The key to an up-to-date test case repository is proactivity. Update it as often as user stories are addressed. With a well ordered test case repository, testers can be easily added to projects. Have you ever wanted to bring in a tester from another project but are reluctant to spend time training them? Up-to-date test cases do the training for you. Assign a new team member to a set of valid test cases and they will know the product in next to no time. On the other hand, if you mistakenly assign a new team member to obsolete test cases with the wrong expected results, there will be nothing but confusion and wasted time. When it comes to test cases, a little work during each sprint saves a lot of work down the road.