Adaptive Filters

CHAPTER 29 Least-Squares Criterion

The earlier parts of this book dealt extensively with the problem of linear least-mean-squares estimation, whereby one random variable is estimated from observations of another correlated random variable. For example, in Sec. 8.1 we studied the problem of estimating a zero-mean random variable d from a zero-mean random row vector u, by seeking the optimal column vector w that solves

(29.1)

The optimal estimator was found to be

(29.2)

in terms of the second-order moments of {d, u}, namely,

The resulting minimum mean-square error was further seen to be given by

where and .

We proceeded in Chapter 8 to devise steepest-descent schemes for evaluating w° iteratively, and in Chapter 10 we showed how stochastic gradient algorithms can be used to approximate w°, also iteratively. These latter algorithms were aimed at situations where access to the moments ...