Common Errors in Statistics (and How to Avoid Them), 4th Edition

FOUR GUIDELINES

In the next few sections on experimental design, we may well be preaching to the choir, for which we apologize. But there is no principle of experimental design, however obvious, however intuitive, that someone will not argue can be ignored in his or her special situation:

• Physicians feel they should be allowed to select the treatment that will best affect their patient’s condition (but who is to know in advance what this treatment is?).
• Scientists eject us from their laboratories when we suggest that only the animal caretakers should be permitted to know which cage houses the control animals.
• Engineers at a firm that specializes in refurbishing medical devices objected when Dr. Good suggested they purchase and test some new equipment for use as controls. “But that would cost a fortune.”

The statistician’s lot is not a happy one. The opposite sex ignores us because we are boring⁵ and managers hate us because all our suggestions seem to require an increase in the budget. But controls will save money in the end. Blinding is essential if our results are to have credence, and care in treatment allocation is mandatory if we are to avoid bias.

Randomize

Permitting treatment allocation by either experimenter or subject will introduce bias. On the other hand, if a comparison of baseline values indicates too wide a difference between the various groups in terms of concomitant variables, then you will either need to rerandomize or to stratify the resulting analysis. Be proactive: ...