Data Science from Scratch, 2nd Edition

Chapter 7. Hypothesis and Inference

It is the mark of a truly intelligent person to be moved by statistics.

George Bernard Shaw

What will we do with all this statistics and probability theory? The science part of data science frequently involves forming and testing hypotheses about our data and the processes that generate it.

Statistical Hypothesis Testing

Often, as data scientists, we’ll want to test whether a certain hypothesis is likely to be true. For our purposes, hypotheses are assertions like “this coin is fair” or “data scientists prefer Python to R” or “people are more likely to navigate away from the page without ever reading the content if we pop up an irritating interstitial advertisement with a tiny, hard-to-find close button” that can be translated into statistics about data. Under various assumptions, those statistics can be thought of as observations of random variables from known distributions, which allows us to make statements about how likely those assumptions are to hold.

In the classical setup, we have a null hypothesis, $H_0$, that represents some default position, and some alternative hypothesis, $H_1$, that we’d like to compare it with. We use statistics to decide whether we can reject $H_0$ as false or not. This will probably make more sense with an example.

Example: Flipping a Coin

Imagine we have a coin and we want to test whether it’s fair. We’ll make the assumption that the coin has some probability p of landing heads, and so our null hypothesis ...