Bayesian Statistics: An Introduction, 4th Edition

3.3 Jeffreys’ rule

3.3.1 Fisher’s information

In Section 2.1 on the nature of Bayesian inference, the log-likelihood function was defined as

In this section, we shall sometimes write l for , L for and p for the probability density function . The fact that the likelihood can be multiplied by any constant implies that the log-likelihood contains an arbitrary additive constant.

An important concept in classical statistics which arises, for example, in connection with the Cramèr-Rao bound for the variance of an unbiased estimator, is that of the information provided by an experiment which was defined by Fisher (1925a) as

the expectation being taken over all possible values of x for fixed θ. It is important to note that the information depends on the distribution of the data rather than on any particular value of it, so that if we carry out an experiment and observe, for example, that , then ...