We’ve all been there… you just released a small change into production and KaBOOM! Customers are writing in unavailable to connect to your app, errors are surfacing, and your team is scrambling to figure out what went wrong and working to roll back changes. Needless to say, things just aren’t going as expected.
In this post, we’re going to continue our series on types of software testing and take a look at regression testing.
So, what is regression testing?
Lets first define the term “regression”. According to dictionary.com…
1. the act of going back to a previous place or state; return or reversion.
2. retrogradation; retrogression.
Also commonly referred to as “verification testing”, this type of testing is the process of verifying that the software still performs correctly after changes were introduced. This type of software testing ensures any enhancements or bug fixes have not adversely affected what was previously built and tested.
When is regression testing needed?
Regression testing comes into play any time production code is changed. This could be from new features being introduced, defects being fixed, performance fixes or improvements, or a change in requirements where an existing feature was modified. Regression tests are ran after a successful deployment.
What is the value of running regression tests?
Executives and managers have a particular interest in regression testing as it gives them the confidence that a change didn’t break what was already there. The development team often depends on this confidence as well so that they can comfortably proceed with other work. Regression testing can often save the company time and money in the long run by finding defects early and fixing them quickly.
How do you do regression testing?
In general, new tests are not created during regression testing but previously created tests are re-executed. However, retesting the entire system and all of it’s test cases can be expensive and very time consuming.
Because of the cost and time involved here, it’s important to strategically select the right tests cases to run for regression testing. Varied strategies and a conscious thought process across the entire team help gain the confidence desired during regression testing.
Some ideas for areas to test during regression testing are:
- Areas exposed to a high number of users
- Areas that have frequent defects
- Core features
- Highly complex functionality
- Areas that have undergone recent changes
- Critical integrations
Automated or Manual Testing?
Since regression testing is typically done after every deployment, it seems easy enough to automate all regression tests, right? After all, automated testing is faster than a human and can be done anytime. So, why not just automated all regression tests?
The truth is, automated tests can’t always catch what a human can catch. Writing and maintaining automated tests can also require a good amount of work to ensure they’re always up to date.
A good rule of thumb is this: If you’re going to automate regression tests, limit automation to the most repeatable tests. Leave the longer, more complex scenarios for manually testing.
Regression testing provides the confidence that your existing software has not been degraded (or downright broken) after pushing changes into production. By giving deliberate and strategic thought in to what is tested, you can make sure you’re not investing too much time on regression testing after each deployment, and covering the areas that are highest risk.