Adaptive Filters

Summary and Notes

The chapters in Part II introduce the basic principles underlying (constrained and unconstrained) linear least-mean-squares estimation with several design examples. The most relevant results are summarized below.

SUMMARY OF MAIN RESULTS

1. Given two zero-mean random variables {x, y}, the optimal linear estimator (optimal in the least-mean-squares sense) of x given y is , where K_o is any solution to the normal equations K_oR_y = R_xy. This construction minimizes the error covariance matrix (or, equivalently, its trace), and the resulting minimum mean-square error matrix is m.m.s.e. = R_x – K_oR_yx
2. The normal equations are always consistent, i.e., they always admit a solution K_o. The solution is unique only when R_y is positive-definite; otherwise there are infinitely many solutions.
3. No matter which solution we pick for K_o (when many solutions exist), the resulting estimator and minimum mean-square error values remain invariant.
4. The optimal linear estimator satisfies the orthogonality condition . That is, the error is orthogonal to the observations, and to any linear transformation of the observations for that matter. In particular, .
5. If the variables {x, y} do not have zero ...