Adaptive Filters

Summary and Notes

This part of the book deals with the basic principles of steepest-descent methods. The key results are the following.

SUMMARY OF MAIN RESULTS

Steepest-Descent Algorithms

1. Consider a zero-mean random variable d with variance and a zero-mean random row vector u with R_u = Eu*u > 0. The solution of the linear least-mean-squares estimation problem
   

   is given by The cost function J(w) = E |d – uw|² is quadratic in w and has a unique global minimum at w°.

2. The optimal solution w° can be determined recursively via a steepest-descent method. Start with any initial guess w_–1 (e.g., w_–1 = 0) and iterate
   

   The successive weight estimates w_i are guaranteed to converge to w° as long as the step-size μ satisfies 0 < μ < 2/λ_max, where λ_max denotes the maximum eigenvalue of R_u. Fastest convergence is attained by choosing μ° = 2/(λ_max + λ_max), where λ_max denotes the smallest eigenvalue of R_u. Moreover, the convergence of w_i to w° is exponential and controlled by the modes {1 – μλ_k} or by the time constants {–1/2 In |1 – μλ_k|}.

3. We can also employ a steepest-descent method with a time-dependent ...