Common Errors in Statistics (and How to Avoid Them), 4th Edition

DETERMINING SAMPLE SIZE

Determining optimal sample size is simplicity itself once we specify all of the following:

• Smallest effect of clinical or experimental significance
• Desired power and significance level
• Distributions of the observables
• Statistical test(s) that will be employed
• Whether we will be using a one-tailed or a two-tailed test
• Anticipated losses due to nonresponders, noncompliant participants, and dropouts

What could be easier?

Power and Significance Level

Sample size must be determined for each experiment; there is no universally correct value. We need to understand and make use of the relationships among effect size, sample size, significance level, power, and the precision of our measuring instruments.

Increase the precision (and hold all other parameters fixed) and we can decrease the required number of observations. Decreases in any or all of the intrinsic and extrinsic sources of variation will also result in a decrease in the required number.

TABLE 3.1. Ingredients in a sample-size calculation

Smallest Effect Size of Practical Interest
Type I error (α)	Probability of falsely rejecting the hypothesis when it is true.
Type II error (1 − β[A])	Probability of falsely accepting the hypothesis when an alternative hypothesis A is true. Depends on the alternative A.
Power = β[A]	Probability of correctly rejecting the hypothesis when an alternative hypothesis A is true. Depends on the alternative A.
Distribution functions	F[(x − μ)σ]; e.g., normal distribution
Location ...