A Panorama of Statistics

16 The pillars of applied statistics II – hypothesis testing

In this chapter, we are looking at hypothesis testing – that peculiarly statistical way of deciding things. Our focus is on some philosophical foundations of hypothesis testing principles in the frequentist, rather than the Bayesian, framework. For more on the Bayesian framework, see CHAPTER 20.

The issues discussed here all relate to a single test. In the next chapter, we investigate some matters that may complicate the interpretation of test results when multiple tests are performed using the same set of data.

Let us begin with a brief refresher on the basics of hypothesis testing.

The first step is to set up the null hypothesis. Conventionally, this expresses a conservative position (e.g. ‘there is no change’), in terms of the values of one or more parameters of a population distribution. For instance, in the population of patients with some particular medical condition, the null hypothesis may be: ‘mean recovery time (μ₁) after using a new treatment is the same as mean recovery time (μ₂) using the standard treatment’. This is written symbolically as H₀: μ₁ = μ₂.

Then we specify the alternative (or ‘experimental’) hypothesis. For instance, ‘mean recovery time using the new treatment is different from mean recovery time using the standard treatment’. We write this symbolically as H₁: μ₁ ≠ μ₂. Though we would usually hope that the new treatment generally results in a shorter recovery time (μ₁ < μ₂), it is conventional, ...