Beautiful Testing

The Last Pieces of the Puzzle

Once the candidate tests pass, the code leaves the test group and is placed in an internal staging environment. We use our own software—the wiki on staging—to essentially run our business.^[91] We use it to create project plans, to track the iteration, to do QA tracking, and to create stories, for blogging and communication. That way, staging becomes the final proving ground for our software before it goes into production, and our employees occasionally do find defects or, more likely, usability issues or performance issues that unit and functional testing did not uncover.

Our entire testing puzzle includes feature testing, wikitests, running test plans, exploratory testing, getting feedback from staging, usability, story review, beta programs, logging and log evaluation, and occasional performance research. If we tried to do everything and do it well, our team would swell in size and make the company unviable—or it would simply burn out the staff very quickly. Instead, we have to ask how much is enough and what can we trade out.

Our goal is not to know exactly what the software can do, but instead to provide a rapid assessment of the software that is fit for purpose. A few key questions I haven’t mentioned yet are: “What should I be doing right now?” and “Am I done yet?”

Again, those are philosophical questions—questions of aesthetics. I would posit that combinations of the puzzle pieces that look the most beautiful are probably the correct ...