Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

F.5 JENSEN'S INEQUALITY

Theorem F.10 (Jensen's inequality). Let X be a random variable defined on an open interval with cdf fX(x), and let g(x) be a convex function. Then

(F.38)

Proof. From the definition of a convex function (see Appendix B), there exists a line c(x) = a(x−xo) + g(xo) such that for all x in the open interval. As a result for random variable X:

(F.39)

Letting in c(X) gives

(F.40)

Substituting this result into (F.39) completes the proof. If g(x) is strictly convex, then (F.38) has strict inequality except when with probability one.

If g(x) is instead a concave function, the inequality in (F.38) is reversed. The following example gives a result in information theory (see Chapter 10) that is also useful for the expectation-maximization algorithm described in Chapter 9.

Example F.3. Consider random variable X with pdf fX(x) and let g(x) be a nonnegative function (which ...