Adaptive Filters

Summary and Notes

In this initial part we highlighted several concepts and results in least-mean-squares estimation theory. Some of these concepts are reproduced below in a less technical language in order to reinforce their importance.

SUMMARY OF MAIN RESULTS

1. The variance of a random variable serves as a measure of the amount of uncertainty about the variable: the larger the variance the less certain we are about the value it may assume in an experiment.
2. The least-mean-squares error criterion is useful in that it leads to tractable mathematical solu tions. The criterion is also intuitively appealing. By seeking to minimize the variance of the estimation error we are in effect attempting to force this error to assume values close to its mean and, hence, to assume small values since the mean is zero.
3. The least-mean-squares estimator of a random variable x given another random variable y is the conditional expectation estimator, namely, . This estimator is optimal in the sense that it minimizes the covariance matrix of the error vector (or, equivalently, its trace), i.e., it solves
   
4. A defining property of the least-mean-squares estimator is that the resulting estimation error is uncorrelated with any function of the observations, namely,

   In particular, and .

5. The evaluation of ...