Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

F.1 CAUCHY–SCHWARZ AND HÖLDER INEQUALITIES

Theorem F.1 (Cauchy–Schwarz inequality). For random variables {X, Y} and functions {g(X), h(Y)}:

(F.1)

Equality is achieved if and only if P(g(X) = ah(Y)) = 1 for some nonrandom a.

Proof. Consider the following mean-square error (MSE):

(F.2)

where is a constant. Differentiating this expression with respect to a and setting the result equal to zero, the expectation is minimum when

(F.3)

Parameter ao is the scalar Wiener solution for this particular MSE (see Chapter 11), and the product aoh(Y) is the minimum MSE estimator for g(X) given measurement h(Y) (see Chapter 9). Substituting ao into (F.2) gives

(F.4)

and

(F.5)

Rearranging this expression, taking the square root of each side, and retaining the magnitude completes the proof. In order ...