Statistical Signal Processing in Engineering

8Cramér–Rao Bound

The Cramér–Rao bound (CRB) [39] [38] sets the lower value of the covariance for any unbiased estimator with parametric pdf that can be asymptotically attained by the ML estimator for a large number of observations.¹ Even if the estimator does not always exist in explicit form and needs numerical optimizations, the CRB can always be computed in closed form and its derivation needs to preserve all the scaling terms in likelihood that are usually neglected in MLE. Often the CRB computation needs care, patience, and some skill in algebra that can be eased by some software tools for symbolic calculus. Proof is provided for the case of a single parameter; it can be extended to any arbitrary set of parameters.

8.1 Cramér–Rao Bound and Fisher Information Matrix

Information indicates the degree of unpredictability of any rv, and the Fisher information matrix is a way to measure the amount of information that an rv x carries about the parameter θ over the parametric pdf dependency . The derivative with respect to the entries of θ of the log‐likelihood function (called the score function) plays an essential role in the derivation of the CRB.

8.1.1 CRB for Scalar Problem (P = 1)

Let be the unbiased estimator with as the true parameter; the variance is lower bounded as ...