Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing

8.7. Case Study

On one project, my associates and I tested an information appliance. This system was comprised of set of a back-end servers to which the clients would connect, giving the users access to the Internet, email, and other services. See Figure 8.10 for an illustration of the system under test, its basic architecture, and the test team.

Starting from the left side of the figure, one thing we wanted to test was actual user experience in the field. For this reason, one of the test engineers worked on usability testing from various outside locations.

Another thing we needed to test was scalability and performance of the servers. As commercial tools were not available to do the type of load generation and probing we needed, we had to write some of the load and performance tests, which talked directly to the servers. The two test toolsmiths were in charge of this effort.

Finally, most of the test execution was actually done by test technicians against the appliances themselves in the test lab. We used primarily scripted manual tests—albeit some were actually manual probes done while performance and load tools hit the servers hard, letting the manual testers know what the user experience would be like at full look. We also did some amount of exploratory testing. For example, the test manager (me) and the appliance and server lead test engineers would spend an hour a day each following our hunches to find bugs and simply using the system the way we thought users would. We often ...