Beautiful Testing

Test Distribution

Once the computers in the test lab are prepared, the next piece (and possibly the most significant piece of the overall system) is to deploy and execute the tests. If you’ve invested in a test lab filled with hundreds of test machines, you want to make sure machine utilization is high—i.e., reducing as much as possible the amount of time those machines are idle, waiting for the instruction to run tests.

The flow chart in Figure 8-2 describes what decisions in test distribution may look like.

Figure 8-2. Test distribution flow chart

Depending on implementation, the first step is to take an inventory of either the test cases or the configurations available for testing. When starting with test cases, the next step is to begin deploying operating system and application configurations to test machines. When starting with available configurations, the secondary step is matching un-run test cases with the current set of machine configurations. In general, the former works a bit better when testing on a known smaller set of configurations, whereas the latter is slightly more convenient when dealing with a larger number of configurations.

One or more machines (sometimes called test controllers) are in charge of distributing the tests to the waiting machines in the test labs. These machines implement a connection system using sockets, message queues, or another multi-machine-aware ...