Adaptive Filters

CHAPTER 11 Normalized LMS Algorithm

Now that we have illustrated the application of adaptive filters (and specifically LMS) in several contexts, we proceed to the derivation of other similar stochastic-gradient methods. In this chapter we motivate the so-called Normalized LMS algorithm.

11.1 INSTANTANEOUS APPROXIMATION

Assume again that we have access to several observations of the random variables d and u in (10.1), say, {d(0),d(l),d(2),d(3),…} and {u₀, u₁ u₂, u₃,…}. The normalized LMS algorithm can be motivated in much the same way as LMS except that now we start from the regularized Newton’s recursion (10.8) and assume that the regularization sequence {∈(i)} and the step-size sequence μ(i) are constants, say, ∈(i) = ∈ and μ(i) = μ

Thus, using

(11.1)

and replacing the quantities (∈I + R_u) and (R_du – R_uw_i–1) by the instantaneous approximations and , respectively, we arrive at the stochastic-gradient recursion

(11.2)

This recursion, in its current form, requires the inversion of the matrix at each iteration. This step can be avoided by reworking the recursion into an equivalent ...