Fundamentals of Deep Learning, 2nd Edition

Chapter 2. Fundamentals of Probability

Probability is a field of mathematics that quantifies our uncertainty regarding events. For example, when rolling dice or flipping a coin, barring any irregularities in the dice or coin themselves, we are uncertain about the result to come. However, we can quantify our belief in each of the potential outcomes via probabilities. We say, for example, that on every coin toss the probability of the coin showing up heads is $\frac{1}{2}$. And on every dice roll, we say the probability of a die facing up with a five is $\frac{1}{6}$. These are the sorts of probabilities we talk about with ease in our daily lives, but how can we define and utilize them effectively? In this chapter we’ll discuss the fundamentals of probability and how they connect to key concepts in deep learning.

Events and Probability

When running a trial such as rolling a dice or tossing a coin, we intuitively assign some belief to the trial’s possible outcomes. In this section, we aim to formalize some of these concepts. In particular, we will begin by working in this discrete space, where discrete signifies a finite or countably infinite number of possibilities. Both rolling a dice and tossing a coin are in the discrete space—when rolling a fair dice there are six possible outcomes and when tossing a fair coin there are two. We term the entire set of possibilities for an experiment the sample space. For example, the numbers one through six would make up the sample space for rolling a fair ...