Black box software testing: by Cem Kaner & James Bach
Risk-Based Testing
- Video lectures
- Part 1 -- Introduction and project risk heuristics [14:35]
- Part 2 -- Failure mode & effects analysis and bug taxonomies [13:02]
- Part 3 -- QuickTests (and attacks) [41:08]
- Part 4 -- Operational profiles, risk-based test management [12:32]
- Part 5 -- Closing summary: the risk-based testing cycle and the scientific method [9:38]
- Lecture slides (PPTs)
- Multiple choice review questions [DOC] [Grading notes]
- Activity for risk-based testing (sorting out the attacks)
- Essay test questions
- Assignment (applying Whittaker's attacks to Open Office)
- Examples
- Bug taxonomies
- Quicktests: Whittaker's Attacks & Interference tests
- Required reading:
(These are required in the Florida Tech course)
- James Whittaker, How to Break Software
- Stale Amland, Risk Based Testing, EuroStar 1999 (Risk-based test management)
- Recommended readings
- Course notes for Bach's Rapid Software Testing course
- Appendices (collected readings) for the Rapid Software Testing course
- Giri Vijayaraghavan & Cem Kaner, "Bugs in your shopping cart: A taxonomy." 15th International Software Quality Conference (Quality Week), San Francisco, CA, 2002. (Best Paper Award at Quality Week.)
- Giri Vijayaraghavan & Cem Kaner, "Bug taxonomies: Use them to generate better tests." Software Testing, Analysis & Review Conference (Star East), Orlando, FL, May, 2003. (Received the Best Paper Award at STAR).
- Paul Gerrard and Neil Thompson , Risk-Based E-Business Testing, (another summary of the risk-based test management approach)
- John Musa, Software Reliability Engineering (2nd Ed) (Operational profiles), author website
- Karl Popper, Conjectures & Refutations
Risk is the possibility of suffering harm or loss. In software testing, we think of risk on three dimensions:
- A way the program could fail
- How likely it is that the program could fail in that way
- What the consequences of that failure could be
Risk-based testing (in my view) focuses on the ways the program can fail--imagine how the program can fail and design tests to trigger those failures. This segment of the course focuses on three classes of heuristics for generating ideas on how programs can fail:
- heuristics that focus on project risks
- failure mode and effects analysis (creating and using failure mode catalogs, bug taxonomies, etc.)
- quicktests-- cheap tests that have some value but require little preparation, knowledge, or time to perform
We are setting up a mailing list for announcements about this course and, perhaps, a tightly focused and moderated discussion of how to teach it or self-study with it. (This won't be a general, high-traffic, intro-to-testing discussion.) If you're interested in the course, please sign up by sending us an email. We will NOT share your email address with third parties or send commercial advertising to you.
We are publishing this course under a Creative Commons license that allows you to freely reuse and distribute the materials and to modify the slides and associated printable materials (but not the videos). We would be appreciate a few mirror sites, to reduce the growing burden on our servers. If you can help in this way, or any other way, please send a note to Cem Kaner.
