What Should Be Included in a Test Summary Report?

Does the world of Agile still need test summary reports? The most common response would be NO, but there are still projects and processes where they remain on the list of deliverables. Apart from waterfall projects, a test summary report still has significance in freelancing. This article provides insight into the importance, and the when and the what of a summary report, from a freelancer’s perspective.

What is a Test Summary Report?

The definition of a Test Summary is as simple as the name suggests. Also known as a Test Closure Report, it provides stakeholders with a condensed account of the overall test results, defects and connected data following a test project.

The Importance of Test Summary Reports for a Freelancer

For freelancers, a test summary report from a previous job of your own can also serve as a template to help you write a proposal-winning example document. Examples are often asked for in freelancing jobs, so being able to demonstrate your overall presentation skills by having a procedure document showing how you would actually write a test summary report, can be very useful. Summary reports also give input on testing coverage, which can be a weighty factor when interacting with potential new clients.

A few weeks back, a client approached me for auditing a project over three days, and they were looking for details on every possible issue. A bug or defect report would have sufficed, but after spending time on auditing the system, we ended up having information and inputs which we could not classify as a bug. We felt that the best way to consolidate our observations under those circumstances was to write a test summary report.

Irrespective of your reason for creating the document (which could be to follow process, or provide a meaningful report with valuable information to further assist in driving decisions), the most critical question here concerns when to prepare, and what to include in the summary report.

When to Create a Test Summary Report?

Ideally, the report is created at the end of the testing cycles, so it can also include regression and retesting. For freelancers, many a time the job will be only to perform one round of QA then provide feedback. In these cases, the summary report must be submitted by the tester after completing their task, which could be just the one cycle of testing with defects in all open status. The intention here must be to help the client with information on the overall health of the application in the test so corrective measures can be planned.

What to Include in the Test Summary Report?

An informative test summary report should aim to be concise and relevant. If you check for templates online, you will find several examples but the metrics might not be applicable in all cases, such as for defect density or defect leakage, due to the nature of the project and work assignment. Therefore, it is vital to customize your report after analyzing precisely what is required and what is not. The best way to do this is to use a template but remove sections from the document that will be left blank if data is not available or applicable, so that the submitted report looks complete.

Here’s a list of what should be included:

1. The objective of the test – Mention the purpose of the testing, this is mainly to indicate that the tester understood the expectation and the requirement.
2. Testing Approach – This is important because it indicates to the client what and how the testing was performed. Provide details of the steps taken and the types of testing done to achieve the task.
3. Areas Tested – This section must highlight all the features and functionalities tested. It doesn’t have to be as detailed as a test scenario but be sure to include high level information if, say, you are testing an eLearning system, and it must outline and verify the flow to add a course, register, pay, and launch.
4. Areas Not Tested – Another significant part of any test summary report is information about any features not tested. Any areas not tested will usually raise an alarm at the client’s end, so make sure that anything has been left untested is noted and expectations are set accordingly. Each point must have an associated reason, which might vary from the limitation of access to the availability of the device, and so on.
5. Platform details – Gone are the days when applications used to be supported just on the web. With the growing demand, testing likewise is not just limited to multiple browsers and devices but different versions as well. So, it is recommended to include details of every single platform and environment tested.
6. Defect Report – Even though this will already be in the bug report, having a link to the same in the summary report can be a plus.
7. Testing Metrics (Optional) – Metrics, like total defects with a breakup of severity, can be helpful. If applicable, include additional metrics regarding test cases,.
8. Overall Summary – This section is mainly a place for you to provide your feedback on the status of the application under test. It must inform the client of all critical issues so they can estimate how close they are to launch and what remediation and time-scale they might need in place for the fix.

Finally

A Test Summary Report is not the only way of communicating a testing summary. Sometimes writing out an email with details can also prove helpful. So, until fully agreed in the contract, it is up to the freelance tester to determine whether to create this report or not, but having a consolidated summary in the form of a report is always an advantage.