Selecting Invalid Games in Freecell

 

Summary

Users can select their own game in Microsoft Freecell by entering the game number that they wish to play. However, some values outside of the given range are accepted as input and generate odd instances of the game.

Application Description

Microsoft Freecell is a card game included with Microsoft Windows.

Test Design

This example demonstrates the use of Domain testing. In Domain testing, we are evaluating a function's acceptance of input by choosing representative data to enter into the data fields.

In Freecell, all of the card arrangements are preset, so users can actually enter a game number to restart/replay a game that they had previous played. When selecting a game in Freecell, the user simply chooses a game by typing in the game number. The user is told that they can choose any game between the numbers 1 and 1,000,000.

As a domain test, we have some logical choices for equivalence classes. We can say that any number between 1 and 1,000,000 should yield the same result, so no need to test more than a few. There are some "magic" numbers, however, to be careful of. Numbers like 1, 128, 65,535 and 1,000,000 are all border cases (meaning that they lie on the edge of a range). 0 and 1,000,001 are also important for that reason. What are some other equivalence classes? We also have letters, symbols, negative numbers, etc.

For this presentation, we are going to look at negative numbers. Our range is 1-1,000,000, so we simply want to verify that the program correctly refutes negative numbers. The best way to perform a domain test is to lay out our possible values in a chart:

 

Variable Risk (Potential Failure) Classes that Should Not Trigger the Failure Classes that Might Trigger the Failure Test (Best Representative) Notes
Number Input Numbers outside of the range of 1-1,000,000 might be accepted Numbers 1-1,000,000

0 and below

1,000,001 and above

-1 Large signed integers loop to negative numbers, so -1 may cover both above and below the range of accepted inputs
Non-Numeric Input No letters or other non-numeric characters should be accepted Numbers Non-numeric characters such as letters or symbols A Any non-numeric character should present the same ability to uncover a bug
White space What if nothing is entered at all? Characters Pressing the 'Enter' key Force the program to accept a null space Clear the input line and press 'Enter'

 

Performing the Test

  1. Open Microsoft Freecell.
  2. Select a game by using Game->Select Game or F3:

  3. Once the Game Number box appears, we can enter a game number by typing it into the text box. Just to see how it works, we can enter a valid number (1-1,000,000) or 0. We find that if we enter a non-numeric input, it simply resets to '0' and allows us to enter another input:

  4. Now let's try a negative number. Again, press F3 or use Game->Select Game and enter "-1" into the textbox:

  5. Press the 'OK' button.
  6. Instead of the selection defaulting to '0', it accepts the input and generates a game:

    Notice how all the cards are sorted by suite in an alternating order: odds are in descending order and evens are in a ascending order.

  7. Again, press F3 or use Game->Select Game and enter "-2" into the textbox:

  8. Instead of the selection defaulting to '0', it accepts the input and generates a game:

    Now the cards are in an ascending order by suite. It might also be worth noting that the aces have been incorrectly sorted, since ace represents a '1' in Freecell.

  9. Lastly, press F3 or use Game->Select Game and enter "-3" into the textbox.

Results/Relevance

For -3 (and any subsequent negative numbers), we are told that it is invalid by defaulting our selection to '0'. However, the program accepted and generated games for -1 and -2. It is difficult to tell whether this is a bug or a hidden feature by the programmers, yet either way, we were told that only numbers 1 through 1,000,000 are valid, meaning that -1 and -2 are bogus input.

Domain testing helped us to focus our testing mindset to look at what we could put into a text box, and to eliminate like input to save time.

Similar Tests/Additional Notes

Often times, bugs such as these are less incidental than they are programmers' "easter eggs" (hidden signatures or features for the programmers to enjoy). Try looking up "easter eggs in programs" with a search engine if you are interested in examples such as the one in this presentation.

Configuration Notes

Testing Microsoft's Freecell v5.1 on:

Created 22 July 2004 for the CSTER

Updated 18 August 2005

All images and written material ©Copyright Sam Oswald 2004

This work is licensed under the Creative Commons Attribution-ShareAlike License.
To view a copy of this license, visit
http://creativecommons.org/licenses/by-sa/2.0/
or send a letter to
Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305,
USA.