Beauty is seeing disparate and rough test tools come together at the right time to create a special testing capability. In this chapter’s case, the test tools were originally developed six years ago when the Mozilla Project (where I am director of quality assurance) was a very different organization. These tools morphed from simple web spidering tools to assertion and crash testers for a hundred web pages to an automated framework for testing one million web pages!
The resultant combined framework was needed to test a huge number of web pages in a very flexible way. This included testing on different operating systems, on different build types (e.g., standard and debug), across hangs and crashes, with highly selectable page sets.
It took the work of multiple people to innovate and collaborate to create interesting new tools along the way. These people built up tools, one upon another. The result was an amazing framework that can collect the top websites into sets. These can be 100 pages, 10,000 pages, or a million-plus pages. They can be the top pages across the world or the top shopping sites in Albania. And they can be tested using standard, debug, or optimized builds. The framework can test any combination of these that your mind can imagine or your test farm can handle.
What follows is the story of how the tools were refactored and morphed to test one million web pages.
…our small Mozilla Project team needed ...