Chapter 2. Topics in Hypothesis Testing
2.1 Significance Levels 23
2.2 Power 25
2.3 One-Tailed and Two-Tailed Tests 26
2.4 p-Values 27
2.5 Sample Size Determination 27
2.6 Multiple Testing 30
2.7 Summary 40
2.1 Significance Levels
When conducting hypothesis testing, an erroneous conclusion is made if the null hypothesis is rejected when it is really true. This error is called a Type I error, and its probability is denoted by α, which is known as the 'significance level' of the test.
When setting up the hypothesis test, the rejection region is selected based on a predetermined value for α, usually a small value such as 0.05. This means that there is only a 5% chance of rejecting a true null hypothesis.
For example, suppose that administration of a drug was ...