Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

9.8 RAO–BLACKWELL THEOREM

The Rao–Blackwell (RB) theorem describes a method for transforming a relatively poor estimator of θ, which is usually straightforward to find, into an improved estimator with lower variance. In order to prove the theorem, we need Jensen's inequality in Appendix F:

(9.111)

where X is a random variable defined on an open interval with cdf fX(x), and g(x) is a convex function (see Appendix B). Although the Rao–Blackwell theorem applies to any convex function as used in Jensen's inequality, we present the theorem for the variance of an estimator.

Theorem 9.5 (Rao–Blackwell). Let be a set of statistics that are jointly sufficient for function of parameter θ. Define another statistic T that is an unbiased estimator of . Then statistic has the following properties.

• is a function of and thus is sufficient.
• so that TRB is an unbiased estimator of .
• for all θ.
• var[ ...