| Welcome | Foundations | Bug Advocacy | Test Design | Exploratory Testing | Taking Exams | Policies | Extras | Instructors Course | Metrics | Engineering Ethics |

Domain Testing: The Most Widely Used Test Technique

Overview

THIS MATERIAL HAS BEEN UPDATED AND INCORPORATED IN THE TEST DESIGN LECTURES AT http://www.testingeducation.org/BBST/testdesign/. WE WILL REMOVE THIS SECTION IN MID-2012.

The essence of domain testing is stratified sampling of a few tests from a huge pool of potential tests.

In domain testing, we partition a domain into sub-domains (equivalence classes) and then test using values from each subdomain.

A domain might involve the values of any one variable or combination of variables. Some books look only at input values, but outputs, intermediate calculations, even configuration variables (such as printer type) are commonly analyzed in practical work in the field.

We define an equivalence class as follows: two values are equivalent if, given your theory of possible error, you expect the same test result from each.

The values that we pick to represent each equivalence class are the most powerful members of each set, the best representatives. A best representative is at least as likely to expose an error as any other member of its set.

There are two learning units on domain testing.

Part 1

This first group of material considers the classical approach, some of the problems applying it, and an alternative, risk-focused way of looking at domain testing.

We often organize the domain analysis in a tabular format. Glen Myers, The Art of Software Testing, describes the traditional format. I prefer this risk-focused one instead.

Variable Risk (Potential Failure) Classes that Should Not Trigger the Failure Classes that Might Trigger the Failure Test (Best Representative) Notes
           

Part 2

This unit starts by breaking away from the simplistic linear model. Configuration testing is the classic example of a large testing space that cries out for an intelligent sampling strategy but that is not linearizable. Kaner, Falk & Nguyen explained how to map this problem to domain testing back in 1993 (second edition of Testing Computer Software). This section explores that approach.

When you look more closely at the descriptions of this sampling strategy in the literature, there are several sets of assumptions that lead the analysis in a slightly different way for simple problems and increasingly divergent ways as problems get more complex. We look at four common sets of assumptions as they relate to the definitions of equivalence classes and best representatives (boundary cases).

Finally, the domain testing discussions constantly focus on independent variables, but the world is full of dependent variables. For example, consider an output variable that is calculated based on several inputs. The output is fully dependent on the inputs. To manipulate the output (drive it to its boundary values, for example), we have to manipulate the inputs because we can't reach the output variable directly. How can we do domain analyses on the output variable?

Slides

  • Domain testing part 1 (PDF)
  • Domain testing part 2 (PDF)
  • Videos

    Lecture 1: The Classical Approach vs. the Risk-Focused Approach

    This first group of material considers the classical approach, some of the problems applying it, and an alternative, risk-focused way of looking at domain testing.

  • Lecture 1a: Intro to domain testing (7 mins) (WMV)
  • Lecture 1b: Classical analysis (10 mins) (WMV)
  • Lecture 1c: Examples (12 mins) (WMV)
  • Lecture 1d: Risk-based equivalence analysis (18 mins) (WMV)
  • Lecture 1e: Summary (4 mins) (WMV)
  • Lecture 2: Breaking Away From the Linear Model

    This unit starts by breaking away from the simplistic linear model. Configuration testing is the classic example of a large testing space that cries out for an intelligent sampling strategy but that is not linearizable. Kaner, Falk & Nguyen explained how to map this problem to domain testing back in 1993 (second edition of Testing Computer Software). This section explores that approach.

  • Lecture 2a: Non-ordered sets (16 mins) (WMV)
  • Lecture 2b: Theory of domain testing (12 mins) (WMV)
  • Lecture 2c: Interacting variables (9 mins) (WMV)
  • Lecture 3: Worked Examples

    Some examples of domain testing.

  • Example: The food van example (8 mins) (WMV)
  • Example: The hockey example (4 mins) (WMV)
  • Example: Glen Myers' example (8 mins) (WMV)
  • Example: Solutions to the examples (9 mins) (WMV)