Pragmatic Software Testing: Becoming an Effective and Efficient Test Professional

Chapter 18. Orthogonal Arrays and All Pairs

In this chapter, I’ll present two more advanced techniques for black-box testing. Both deal with the same test problem, that of combinations where factors should not interact but might. However, each technique solves the problem in a slightly different way.

Combinatorial Explosions

If you didn’t go through the chapter on domain testing (Chapter 16), let me revisit the problem. Sometimes, the number of possible test cases becomes very large due to the number of variables or factors and the potentially interesting test values or options for each variable or factor.

For example, suppose you have 10 integer input fields that accept a number from 0 to 99. There are 10 billion billion valid input combinations.

Equivalence class partitioning and boundary value analysis will reduce but not resolve the problem. You have four boundary values for each field. The illegal values are easy, because you only have 20 tests for those. However, to test each legal combination of fields, you have 1,024 test cases.

But do you need to do so? And would testing combinations of boundary values necessarily make for good tests? Are there smarter options for dealing with such combinatorial explosions?

There are two general cases:

• The factors should interact. One or more factors should partition the system’s behavior into two or more domains, situations in which the system processes data equivalently. In addition, one or more factors interact in the system data processing. ...