| Welcome | Foundations | Bug Advocacy | Test Design | Exploratory Testing | Taking Exams | Policies | Extras | Instructors Course | Metrics | Engineering Ethics |
THIS MATERIAL HAS BEEN UPDATED AND INCORPORATED IN THE TEST DESIGN LECTURES AT http://www.testingeducation.org/BBST/testdesign/. WE WILL REMOVE THIS SECTION IN MID-2012.
I define Specification-Based Testing broadly to include testing
against any published claim about how the program is supposed to
work. Thus, we might test against a comprehensive development
specification, a user manual, a third-party manual, or a collection of
emails and meeting notes that together define the product.
Specification-based testing is a style of testing (a style is a
collection of test-related activities and techniques) focused on
- discovering what claims are made in the specifications and
- testing the product against them.
Course sections on spec-based testing often focus on analyzing fine
details of the specification, such as ambiguity analysis or teasing
out the logical relationships among variables to support combination
testing. These are testworthy issues, but I think the biggest
challenge in specification-based testing is figuring out what the
specification says. Specifications are often hundreds or thousands of
pages long. Organizing and reconciling its many treatments of the same
issues, and its remarkably light treatments of some other issues,
requires a conscious, active interaction with the set of documents
that together make up the specification.
This section of the course considers the following questions:
- What is the specification?
- Of the many documents labeled "specification," which of the
conflicting assertions in them are the correct (currently
intended) descriptions of the product? There are also other
documents, implicit specifications, that were not written as
specifications of this particular product but that describe
aspects of how the product must work (or how people will
expect it to work).
- What does the specification say?
- We review active reading techniques for identifying and
organizing complex sets of material.
- This includes review of questioning techniques, including
- We also apply the Heuristic Test Strategy Model to organize
analysis of a complex specification.
- What are some techniques for critiquing the specification?
- This is where we cover ambiguity analysis
- How to critique the specification?
- We look briefly at the process of conducting specification
- How to derive tests from the specification?
- The main issue we consider here is coverage of the
specification, including use of the traceability matrix
- What legal issues can help guide us in testing the
- Written statements of fact in documents created by the vendor
of a product are often interpreted as warranties--legally
binding promises that the product will perform as
described. Despite the potential legal liability, many
software publishers invest minimal time checking the accuracy
of their documentation.
Lecture slides (PDF)
Lecture 1 (22 mins) (WMV)
Lecture 2 (25 mins) (WMV)
Lecture 3 (17 mins) (WMV)
Lecture 4 (22 mins) (WMV)