Controlled Experiments and Quasi-Experiments 221
is just that. Indeed, a significant result is not necessarily an important one.
Given enough participants, it is usually possible to obtain a low value of α
even when the effect of the treatment is not very large, and hence less likely
to be of interest.
The arbitrary nature of significance levels, the ability to change signifi-
cance levels by increasing the number of participants, and the disconnect be-
tween the significance of the results and their importance, are the reasons why
many statisticians are strongly opposed to the standard null hypothesis test-
ing framework described above. As an alternative, they recommend reporting
effect sizes (which we discuss in the context of meta-analysis in Chapter ...