Black box software testing: A course by Cem Kaner & James Bach
Introduction to test design
- Video lecture: First half [18:33]
- Video lecture: Second half [21:17]
- Video exercise [7:40]: This is a standalone discussion of the mission of the test group, set in the context of a comparison of two groups testing the same program in two different ways. The full exercise should take about 15-20 minutes of class time.
- Lecture slides (PPTs)
- Essay test questions
- In-class exercise: (mapping attributes to the main test techniques)
- Required readings
(this is what I require at the Florida Tech course)
- Kaner: What makes a good test case
- Teasley, Leventhal, Mynatt & Rohlman: Why software testing is sometimes ineffective: Two applied studies of positive testing strategy (NOTE: unfortunately, this is not on the open web, you have to get it through one of the databases. It's an important counter to the constant blathering we get from some people that testers who take a negative testing approach--attempting to prove the program is broken--are doing the wrong thing, not being team players, and not any more likely to find worthwhile information abou the program. I don't expect papers like this--or any amount or kind of scientific research--to convince some of the well-known folks on the conference circuit. But the rest of us can come to our own view.
- Kaner, Bach, Pettichord, Lessons learned in software testing, Chapter 3 on test techniques.
- Recommended reading
- Lee Copeland: A practitioner's guide to software test design. One of the best testing books published in this decade.
- Bach, A framework for good enough testing
- Whittaker: What is software testing? And why is it so hard?
Test design involves creation of high-quality tests to help you discover and interpret the information that you want to discover.
- The study of test techniques is the study of the kinds of tests you can create
- Given a variety (a huge variety) of techniques, we ask about quality. How are tests created with this technique better or worse than another technique's? There are several desirable attributes of tests, such as power (ability to reveal a bug) and credibility (extent to which people will say this is a realistic reflection of the ways the program will be used or abused). Some techniques (e.g. domain testing) are more oriented to power, others (scenarios) more to credibility. Each technique has its own profile of strengths and weaknesses.
- Test groups has different missions, or over the short term, different information-seeking objectives. A group that's trying to find lots of bugs quickly will look for bugs differently from a group that's trying to assess whether the program meets a specification. Not surprisingly, the relevant test techniques differ too.
Finally, there are so many test techniques that we need a model to help us make sense of them. I overview in the lectures the model that Bach and Pettichord and I presented in Lessons Learned.Newcomers to testing won't find this discussion helpful. Old hands who've worked with a dozen or more techniques and read about dozens more will (we think) find this a useful way of reorganizing that knowledge and experience about the nature and uses of those many techniques.
We are setting up a mailing list for announcements about this course and, perhaps, a tightly focused and moderated discussion of how to teach it or self-study with it. (This won't be a general, high-traffic, intro-to-testing discussion.) If you're interested in the course, please sign up by sending us an email. We will NOT share your email address with third parties or send commercial advertising to you.
We are publishing this course under a Creative Commons license that allows you to freely reuse and distribute the materials and to modify the slides and associated printable materials (but not the videos). We would be appreciate a few mirror sites, to reduce the growing burden on our servers. If you can help in this way, or any other way, please send a note to Cem Kaner.
