Anyone with children (or in my case, nieces) will know that they spend their days trying to make sense of new experiences in the world around them. They explore, they discover and without even realising it, they employ The Scientific Method.

The scientific method to software testing

Almost a thousand years before the birth of modern computing, the polymath Ibn al-Haytham, advocated the importance of asking questions and then testing them. Galileo built on that and detailed his experimental methods in 1638 with the publication of “Two New Sciences”. He believed that knowledge could be acquired through reason, then reinforced through observation and experimentation. The Scientific Method has been honed for this very purpose and here, I will cover the steps involved. These steps are not set in stone and would be best considered as a set of general principles to work by; use them as another tool, not a straightjacket.

6 Steps of the Scientific Method

Make Observations

“You can observe a lot by just watching.”

Yogi Berra

Testers work in a similar manner, ferreting out the truth from the myths and outright lies. I like to start right at the top of that diagram. I just dive right in, begin testing and making my own observations on the fly. This can be anything and everything from overall functionality to specific functions or aspects, tools and widgets, to page layout, look and feel, ease of use and readability.

Interesting Questions

All the time you observe, think of interesting questions. Are page load times acceptable? Does JavaScript slow things down unacceptably? Is the page or site navigable? Does that web form function as expected? Is this page too graphics heavy? I once tested a website with a very high-resolution BMP image in the background that doubled the size of every page it was used on and yes, it was used on every… single… page.

Formulate Hypotheses

Keep asking questions and you will start to get an idea of what is happening with the system under test so you then have hypotheses to work with. Depending on how much access to the underlying code you have, you may have a very good idea of the internal goings-on. Perhaps you just take a black box approach to testing.

Testable Predictions and Gathering Test Data

Whichever way you have observed system behaviour, you will then know that you can make an intelligent, informed and testable prediction (as opposed to just a wild guess) about how the system behaves, and will behave, given a specific set of inputs.

Gathering data is probably the part of this method testers are most familiar with and spend the most time doing. You test the testable prediction, push the envelope of your hypothesis and do everything you can to either prove or disprove yourself. I’m not sure which is the most satisfying event as a tester – that “Eureka!” moment when something turns out exactly as you predicted during a test, or the instance something unexpected happens and you wonder out loud “Hmm, that’s funny…”?

Be ready to adapt your Hypothesis

The reality of testing falls somewhere between the two stools mentioned above. At those all too frequent points, you get up, rub your bruised elbows and have a serious think about what’s happened while you refine, alter, expand or reject your hypothesis. It may just need a tweak or two, or you might have to go right back to the drawing board (or Kanban board).

Try not to lose heart if you find yourself back at square one, because it’s not really square one any more. Every test you do will in some small way add to the sum of your system knowledge. As you work and document the testing processes and results, you are creating your own knowledge base built on all of your observed results.

Develop General Theories

The payback at the end of all your extensive testing work is that you should now be able to develop general theories about the system under test. Once you get this far, testing should, in theory, get easier because by this point, you’re not a mere tester, but have become a product expert.

Now you have a general working theory, you can make even better observations, with greater insight and your predictions are likely to be more refined and accurate. Exceptions to the general rules that you’ve hammered out will be much easier to spot. You’ll go back to making observations again, but this time with more testing experience, confidence, knowledge and wisdom.

Conclusion

Consciously or unconsciously, we all adopt these processes to some degree in our daily lives, but The Scientific Method only gets serious mileage and attention in fields where the pursuit of knowledge is the primary goal. Kids are like sponges, absorbing as much information as they possibly can every second of every day – Scientists and Testers can learn a lot from that.