Bayesian Statistics: An Introduction, 4th Edition

3.10 Approximations based on the likelihood

3.10.1 Maximum likelihood

Suppose, as usual, that we have independent observations whose distribution depends on an unknown parameter θ about which we want to make inferences. Sometimes it is useful to quote the posterior mode, that is, that value of θ at which the posterior density is a maximum, as a single number giving some idea of the location of the posterior distribution of θ; it could be regarded as the ultimate limit of the idea of an HDR. However, some Bayesians are opposed to the use of any single number in this way [see Box and Tiao (1992, Section A5.6)].

If the likelihood dominates the prior, the posterior mode will occur very close to the point at which the likelihood is a maximum. Use of is known as the method of maximum likelihood and is originally due to Fisher (1922). One notable point about maximum likelihood estimators is that if is any function of θ then it is easily seen that

because the point at which is a maximum is ...