Statistics for Data Science and Analytics

4Accounting for Chance—Statistical Inference

In this chapter we connect probability to the process of drawing conclusions from data. After completing this chapter, you will be able to:

1. Explain the rationale for why hypothesis tests are needed
2. Identify when hypothesis tests are appropriate
3. Distinguish the circumstances that call for formal vs. informal hypothesis tests
4. Explain the logic of a hypothesis test
5. Interpret the results of a hypothesis test

The task of trying to assess the impact of random variability on the conclusion from a study, or the results of a measurement, is called statistical inference. In this chapter we will look at a particular kind of statistical inference called a hypothesis test. Generally, a hypothesis test seeks to determine whether the effects we see in some data from a study are real or might just be the result of chance variation. The logic of a hypothesis test runs as follows:

Variation from random chance is everywhere, and we are easily fooled into thinking it might be meaningful. When we see a data pattern or effect that we think is important and real, e.g. a new medical treatment produces better health outcomes or a new web ad being tested produces more clicks, we set up a chance model (flipping coins or drawing numbers) to see whether it can produce a result as unusual as the pattern or effect that we actually saw.

4.1 Avoid Being Fooled by Chance

Why does hypothesis testing, perhaps the most confusing and controversial aspect of statistics, ...