Beautiful Testing

Adding Different Perspectives

There was a day when the three of us, Michael, Ed, and I, were talking in a hallway. We were talking about how each of us felt pretty certain there were still defects in the product. Even though the test team had been diligent in testing, something was missing. Where was our confidence and conviction that the product was ready? We each had the sense that there were issues still lurking. It was more than a hunch; we each knew of a few bugs that hadn’t been consistently reproduced and corrected. Each of the testers felt the same: the product wasn’t ready yet.

How could we solve the problem?

From a business perspective, Ed worked to get the approvals and financing needed to implement our new ideas. It’s easy not to think about what efforts Ed had to work through with management, because he gracefully kept those concerns from draining our energies. Michael and I remained focused on the testing efforts.

Like many complex problems, it took more than one solution. First, we added a different approach to testing. We wanted an opportunity to follow through on our hunches where issues remained. A new team of testers would be brought in to execute testing in an exploratory approach while the existing testers who had been working with prepared written test scripts would continue. The plan was to keep the scripted testers executing the test scripts that would be required for the FDA audit before product launch and to have the second team find what was missing. The ...