Mastering Machine Learning Algorithms

An example of parameter estimation

In this example, we see how it's possible to apply the EM algorithm for the estimation of unknown parameters (inspired by an example discussed in the original paper Maximum likelihood from incomplete data via the em algorithm, Dempster A. P., Laird N. M., Rubin D. B., Journal of the Royal Statistical Society, B, 39(1):1–38, 11/1977).

Let's consider a sequence of n independent experiments modeled with a multinomial distribution with three possible outcomes x₁, x₂, x₃ and corresponding probabilities p₁, p₂ and p₃. The probability mass function is as follows:

Let's suppose that we can observe z₁ = x₁ + x₂ and ...