Hands-On Mathematics for Deep Learning

Bayesian estimation

Throughout this section on statistics, we have dealt with what is known as the frequentist approach. Now, however, we will look at what is known as the Bayesian approach, where we treat θ as a random variable, we tend to have prior knowledge about the distribution, and, after collecting some additional data, we find the posterior distribution.

Formally, we define the prior distribution as a probability distribution of θ before collecting any additional data; we denote this as π(θ). The posterior distribution is the probability distribution of θ dependent on the outcome of our conducted experiment; we denote this as π(θ|x).

The relationship between the prior and posterior distributions are as follows:

Generally, we avoid ...