Multiple comparisons arise in several ways in clinical trials; common examples include multiple arms, multiple end points, subgroup analyses, and monitoring over multiple time points. At first, it seems that trying to answer several questions in the same trial is very efficient, especially if another large trial to answer the same questions is unlikely. The problem is that if the *comparisonwise error rate* (also known as the *per-comparison error rate*)—the expected proportion of false positives—is 0.05, then the chance of at least one false positive, which is known as the *familywise error rate* (FWE) or the *experimentwise error rate*, can be substantially greater than 0.05. In fact, with enough comparisons, the FWE can be close to 1. To control the FWE at level 0.05, one must require stronger evidence for each individual comparison; this is called a *multiple comparison adjustment*.

Two different levels of control of the FWE exist. The first level, which is called *weak control*, means that under the global null hypothesis that each separate null hypothesis is true, the probability of at least one type 1 error is no greater than α. But the global null hypothesis is a very strong assumption; it is more likely that some null hypotheses are true and others are not. We would like the probability of rejecting at least one true null hypothesis to be α or less, where that probability ...

Start Free Trial

No credit card required