What is Non-Functional Testing? – All you need to know

Non-functional testing focuses on user expectations, whereas functional testing is to do with user requirements. Non-functional testing examines how a system operates rather than looking at specific functions or behaviors.

This article discusses and defines non-functional testing, how it is performed and offers a roundup and links to tools that can help.

Why is Non-Functional Testing necessary?

When we receive a new feature to test in QA, we are often given information like acceptance criteria and requirements to build our test cases around. We write happy paths and failure cases so we can try the new feature from different aspects. Then, we would check it against the regression suite. Often, due to time restraints, this is the only testing possible before the next feature is ready to test.

Apart from these necessary tests, there is much more to QA when assessing a new feature. To reach better test coverage for a product, we should examine the whole system.

Both functional and non-functional tests are essential. For example, imagine testing a login for a new site. The tester enters a valid username and password, clicks Enter but has to wait two minutes before being logged in. The login function may work, but the delay would not meet a user’s expectation. What causes the long processing time? By performing some non-functional tests, QA could determine the root cause.

Risks of not Performing Non-Functional Testing

If non-functional testing is not performed, the product is at risk of failure. Data from non-functional tests can define optimal to stressful loads on a system. Without this data, unexpectedly heavy use could cause a crash. Alternative test results could provide insight into security vulnerabilities that hackers might use to infiltrate a system. It is unusual for these types of risks to be discovered through functional testing.

Types of Non-Functional Testing

Non-functional testing is a form of white box testing, meaning the tester operates from the basis of knowing how the system operates internally or structurally. There are many types of non-functional tests, and their particular significance will depend upon the project.

• Performance testing – is a general term for describing how a system performs during different usage, most often by using the two specific tests listed below:
  • Load testing – is performed by applying regular operations under normal circumstances, then monitoring the system’s performance as conditions are amplified.
  • Stress testing – is performed by taking the standard conditions, then incrementally adding a heavier load to determine how the system performs under stress.
• Security testing – Testing that aims to reveal vulnerabilities that could potentially be open to abuse. A Cybersecurity team will often perform these tests.
• Compatibility testing – Testing an application across multiple browsers, operating systems, devices, networks, or versions to determine any notable discrepancies.
• Installation/ Upgrade testing – Installing or upgrading/updating a program and testing that the process works without issues.
• Compliance testing – Determining if a system meets certain required technical standards, legal requirements, and/or contractual expectations.
• Accessibility testing – Testing to determine how well a system can support the needs of users with specific disabilities.
• Disaster Recovery / Failover testing – Following each step of a disaster recovery plan to ensure the system will restore operations satisfactorily in the event of a disaster or failure.

Test Planning for Non-functional Testing

You may be wondering where non-functional testing will fit in your test plan. Some companies have a product-readiness procedure that may include some non-functional testing for completion before a system goes live. If a project is already up and running, you may want to allocate time to perform some of these tests during upcoming sprints.

Non-Functional Testing Tools

There are a range of tools available for non-functional testing. These tools each address a specific testing need to help the tester get the data they require.

Performance Testing Tools

• Apache JMeter – is a highly popular performance testing tool. It’s an open-source program with extensive documentation. Finding free tutorials for specific scenarios is straightforward.
• LoadNinja – by SMARTBEAR is another popular tool. It has a modern UI with easy-to-understand reports and integrates with Jenkins so you can run load tests every time you deploy. It also claims to have a 60% reduction time in test script creation. The subscription is pricey, however.

For additional tools, please see our list of the top performance testing tools.

Security Testing Tools

• Zed Attack Proxy (ZAP) – by OWASP is an open-source web scanner with a user-friendly GUI for beginners or a command-line option for more advanced users.
• Wfuzz – an open-source security tool that uses brute force to expose vulnerabilities. Suitable for a more advanced user.
• SonarQube – a code quality and security tool designed for developers. It integrates with Jenkins, Azure, Bitbucket, GitHub, and more, to ensure clean and safe code. It has open-source, free options as well as paid versions.

Compatibility Testing Tools

• Browserstack – offers an out-of-the-box option for testing apps and browsers on more than 3000 real devices.

For further information and detailed reviews of more tools, please see our list of cross-browser testing tools.

Accessibility Testing Tools

• WAVE – is a free option that uses the Web Content Accessibility Guide (WCAG) standards to find the most common errors by either entering the URL in the website’s tool or using the browser extensions available for Firefox and Chrome.
• SortSite – checks against WCAG and Usability.gov standards. It claims that just by entering the URL, it will check an entire site against 1300 standard-based checkpoints. It also checks for broken links and browser compatibility.

Challenges when performing Non-Functional testing

Non-functional testing, like other testing methods, can come with challenges. For example, non-functional requirements (NFR) can be notoriously unclear, which can make it less than straightforward to perform non-functional tests. Time gets wasted when the QA has to seek out more information because of vague requirements. One way to alleviate this challenge is to take action during sprint and technical planning by asking the Project Manager to provide specific details. Ask for the exact information you need.

There can also be issues to overcome when you need to rely on other departments to complete tasks before performing your testing. For example, QA may have to wait on DevOps to create an acceptable environment for performance testing. Likewise, they may have to wait for the Cybersecurity team to run a security before you start security testing. If this occurs, you can label the ticket as ‘Blocked’ with a description of why and continue to update it with who you spoke to about unblocking it. Or, you can go to your Scrum Master or Project Manager for assistance and let them know that you are waiting and why.

Learning a new program to be able to perform non-functional tests can also be burdensome. Some programs require knowledge of a specific coding language that you may have to learn. Luckily, free tutorials are readily available online. Block out time on your calendar, or schedule a meeting with only yourself, so that you are devoted to the learning process. Also, remember to consider the learning curve when estimating story points or completion times.

Conclusion

Non-functional testing is an important way to improve the user experience of your product because the method tests the system’s operations. It may be challenging to find the time or to learn a new program, but it is beneficial nonetheless. There are several types of non-functional testing, and each will not necessarily apply to every project. However, including non-functional testing in your test plan is essential for achieving fuller test coverage and a better product.