R in a Nutshell, 2nd Edition

Experimental Design Example

Suppose that you want to test the efficacy of a new drug for treating depression. A common score used to measure depression is the Hamilton Rating Scale for Depression (HAMD). This measure varies from 0 to 48, where higher values indicate increased depression. Let’s consider two different experimental design questions. First, suppose that you had collected 50 subjects for the study and split them into two groups of 25 people each. What difference in HAMD scores would you need to observe in order for the results to be considered statistically significant?

We assume a standard deviation of 8.9 for this experiment.^[53] We’ll also assume that we want a power of 0.95 for the experiment (meaning that the probability of a Type II error is less than 0.05). To calculate the minimum statistically significant difference in R, we could use the following expression:

> power.t.test(power=.95, sig.level=.05, sd=8.9, n=25)

     Two-sample t test power calculation 

              n = 25
          delta = 9.26214
             sd = 8.9
      sig.level = 0.05
          power = 0.95
    alternative = two.sided
 NOTE: n is number in *each* group

According to the output, the difference in means between the two groups would need to be at least 9.26214 to be significant at this level. Suppose that we doubled the number of subjects. What difference would be considered significant?

> power.t.test(power=.95, sig.level=.05, sd=8.9, n=50) Two-sample t test power calculation n = 50 delta = 6.480487 sd = 8.9 sig.level = 0.05 power = 0.95 alternative ...