Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing

A.16. Pilot Testing

Once software is written and run through final system and perhaps acceptance testing, production is often as simple as copying bits onto a CD-ROM or floppy, or posting the software to a Web site. However, there is a distinct difference in hardware between skilled engineers hand-assembling an engineering prototype system and multiple people on a semi-automated assembly line doing the same thing for dozens or hundreds of systems per hour. Production of hardware systems is a complex endeavor. The quality of the outgoing products is influenced by the quality of the components and subsystems, the assembly process, the production environment, the training given to the assembly line team, and potentially myriad other factors.

To figure out what these factors are and whether the vendor is able to manufacture systems of an acceptable quality level and in an acceptable volume, a pilot test is often run during the initial production period. During the pilot test, heavy sampling and testing of the systems—sometimes as high as 100 percent—is performed. The appropriate period for burn-in testing (i.e., tests run to screen out those systems that will suffer infant mortality) is determined. I have sometimes been involved in this testing, even though my official role was as the development system test manager.