Article by: Bertrand Meyer
While everyone knows the theoretical limitations of software testing, in practice we devote considerable effort to this task and would consider it foolish or downright dangerous to skip it.
Other verification techniques such as static analysis, model checking, and proofs have great potential, but none is ripe for overtaking tests as the dominant verification technique. This makes it imperative to understand the scope and limitations of testing and perform it right.
The principles that follow emerged from experience studying software testing and developing automated tools such as AutoTest.
Principle 1: Definition
To test a program is to try to make it fail.
Principle 2: Tests versus specs
Tests are no substitute for specifications.
Principle 3: Regression testing
Any failed execution must yield a test case, to remain a permanent part of the project’s test suite.
Principle 4: Contracts as oracles
Oracles should be part of the program text, as contracts. Determining test success or failure should be an automatic process consisting of monitoring contract satisfaction during execution.
Principle 5: Manual and automatic test cases
An effective testing process must include both manually and automatically produced test cases.
Principle 6: Empirical assessment of testing strategies
Evaluate any testing strategy, however attractive in principle, through objective assessment using explicit criteria in a reproducible testing process.
Principle 7: Assessment criteria
A testing strategy’s most important property is the number of faults it uncovers as a function of time.
Loading ...