A Course in Black Box Software Testing
Examples of (In)Complete Testing
See lecture notes on the impossibility of complete testing.
Copyright (c) Cem Kaner, 2004
Complete testing must mean that, at the end of testing, you know there are no remaining unknown bugs. After all, if there are more bugs, you can find them if you do more testing. So testing couldn't yet be "complete."
Complete testing is impossible, but people try to measure the completeness of their testing in several ways, such as:
- coverage (percentage tested of statements, branches, conditions or other countable aspects of the product).
- bug find rates (such as new bugs found per week, bugs still open, ratio of bugs found to bugs fixed)
- predictions from reliability models .
Here are examples that illustrate the difficulty of complete testing or ways to compute some aspect of the extent of testing of the program:
Copyright (c) Cem Kaner 2004
This work is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.
These notes are partially based on research that was supported by NSF Grant EIA-0113539 ITR/SY+PE: "Improving the Education of Software Testers." Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
