Data Science from Scratch

Chapter 6. Probability

The laws of probability, so true in general, so fallacious in particular.

Edward Gibbon

It is hard to do data science without some sort of understanding of probability and its mathematics. As with our treatment of statistics in Chapter 5, we’ll wave our hands a lot and elide many of the technicalities.

For our purposes you should think of probability as a way of quantifying the uncertainty associated with events chosen from a some universe of events. Rather than getting technical about what these terms mean, think of rolling a die. The universe consists of all possible outcomes. And any subset of these outcomes is an event; for example, “the die rolls a one” or “the die rolls an even number.”

Notationally, we write to mean “the probability of the event E.”

We’ll use probability theory to build models. We’ll use probability theory to evaluate models. We’ll use probability theory all over the place.

One could, were one so inclined, get really deep into the philosophy of what probability theory means. (This is best done over beers.) We won’t be doing that.

Dependence and Independence

Roughly speaking, we say that two events E and F are dependent if knowing something about whether E happens gives us information about whether F happens (and vice versa). Otherwise they are independent.

For instance, if we flip a fair coin twice, knowing whether the first flip ...