Common Errors in Statistics (and How to Avoid Them), 4th Edition

Chapter 4

Data Quality Assessment

Space Shuttle Challenger exploded on January 28, 1986, as the world watched in horror. Could this explosion have been avoided? Only if management had heeded the statisticians.

Prior to launch, the risk of a catastrophic failure in the shuttle was estimated by NASA management at 1:100,000. Engineers put that risk at between one in 100 and one in 300.

When statisticians analyzed the same figures afterwards they calculated the actual risk of disaster to be 12–14 percent or about one chance in eight.

—Statistical Society of Australia Inc. (SSAI)

JUST AS 95% OF RESEARCH EFFORTS ARE DEVOTED to data collection, 95% of the time remaining should be spent on ensuring that the data collected warrant analysis.

A decade ago, Dr. Good found himself engaged by a man he had met at a flea market to consult for a start-up firm. The pay was generous but it was conditional on the firm receiving start-up capital.

Laboring on a part-time basis for six months, Dr. Good was concerned throughout both by the conditional nature of the pay and by the tentative manner in which the data were doled out to him. “May I see the raw data?” he kept asking, but each time was told by his sponsor that such a review was unnecessary.

One day, the President of the firm, heretofore glimpsed only from a distance, called Good into his office and asked for a summary of the results. Good responded with a renewed request for the raw data.

With marked reluctance, and only after nearly a week ...