Hands-On Mathematics for Deep Learning

Likelihood

Generally, in practice, when we want to determine whether or not our estimator is good, we usually use the maximum likelihood estimator (MLE).

Given n random variables (with being the joint PDF), then if X = x, the likelihood of θ is defined as . Therefore, the MLE of θ is an estimate of the value of θ that maximizes .

In practice, however, we maximize the log-likelihood instead of simply the likelihood.

Going back to the example ...