Recently, some leaders in the testing field suggested that all testing is in fact, exploratory testing. Testing methodology was actually very informal beginning in the 1960s, and it was not until the 1970s that more formalized standards and test scripts came into vogue. But formalization does not always equal improvement, and with the growth of agile methodology, proponents of exploratory testing have high praise for a more flexible testing approach.
In exploratory testing, test planning, test design and test execution occur in unison. By not having to script out the test in advance, exploratory testing puts more discretion at the hands of the tester. By not limiting the tester’s thought process to pre-determined documentation, we can empower the tester to take a more fluid approach to finding problems quickly.
The beauty of exploratory testing is that it is only limited by the testers’ imagination and insight into the application. As long as the tester has the skills to listen, read, think and report accurately and effectively, exploratory testing can be very productive and produce results not anticipated by a script.
How exploratory testing makes agile work
This ability to react and adapt to change quickly makes exploratory testing a very strong player in agile. With the fast pace of an agile environment, the test rarely has the luxury of receiving complete documentation that can be relied on to write a scripted test scenario. By comparison, exploratory testing enables expert testers to go into the product with only a general idea of how the functionality works and become dedicated advocates for the end user.
Exploratory testing also enables test planning, test design, and test execution to happen in parallel. That means testing teams can keep pace with a driven and adaptive agile environment.
As automation frameworks become more advanced and prevalent, manual scripted testing becomes a useful—yet less pervasive element—to the testing process. However, the creative nature of exploratory testing will always need to be human driven.
How exploratory testing makes a better product
An exploratory test session typically begins with a charter that defines the mission. The charter may be defined by tester or assigned by a test lead. If the mission is clearly focused on the needs of the end user, then the resulting product will be one with maximized potential to attract the targeted customer base.
In defining questions to support the mission, the tester can help to continually refine and improve the product. The tester develops questions about the product and then designs tests to answer those questions. If the questions are not satisfied by the test results, then the tests are tweaked until results provide insight into accomplishing the mission.
Here is an example of a testing charter based on a sample published by James Bach, an expert in test design:
- Explore and analyze the elements of the product. Produce a test coverage outline.
- Identify and test all claims in the product manual.
- Define workflows through the product and try each one. The flows should represent realistic scenarios of use, and they should collectively encompass each primary function of the product.
- Understand the performance and reliability characteristics of the product as complexity increases. Start with a nominal scenario and scale it up in terms of number of options and factors until the application appears to hang, crash, or prevent user from moving further.
- Test all fields that allow data entry.
- Analyze product performance in a specific file format and determine the behavior of the application when its elements are programmatically manipulated. Test for error handling and performance when coping with pathological scenario files.
- Check UI against Windows interface standards.
- Determine if there any way to corrupt a scenario file. How would you know it’s corrupted?
- Investigate the feasibility of writing an automatic file checker.
- Test integration with external applications.
- Run the product under AppVerifier and report any errors.
How Exploratory Testing and Automated Testing can work together
Exploratory and automated testing should not be an either/or proposition and in fact can work to support one another. If exploratory tests are recorded, they can be converted to manual or automated tests. Automated tests can also provide results that drive future exploratory ideas. If an automated test fails, it can drive an exploratory effort to regress back to find the issue. If the automated test succeeds, it can still provide insight into areas that need further exploration.
Exploratory testing is here to stay
Exploratory testing is such an essential component of any testing process; does it even need to be called out by name? Where fixed, scripted testing methods were once considered the only “accepted” way to test, we now know that such rigidity is a way to script our own failure. The free flow of ideas has been proven time and again to be the cornerstone of successful businesses across almost every industry. Why would the field of testing be any different?