Adaptive Filters

CHAPTER 12 Other LMS-Type Algorithms

The idea of using instantaneous approximations to devise stochastic-gradient algorithms from steepest-descent implementations is not limited to quadratic cost functions as in (10.1). For instance, in Probs. III.16–III.21 we formulate steepest-descent methods for a variety of other cost functions. If we then employ instantaneous approximations for the associated gradient vectors and Hessian matrices, we would obtain other well-known adaptive algorithms. In this chapter, we list the recursions for various such algorithms of the blind and non-blind types. Non-blind methods are so-called because they employ a reference sequence {d(i)} in their update recursions. On the other hand, blind algorithms do not use a reference sequence.

12.1 NON-BLIND ALGORITHMS

We first list several non-blind methods derived in Probs. 111.26–111.31. The derivations in the problems lead to the following statements for the so-called sign-error LMS, leaky-LMS, least-mean-fourth (LMF), and least-mean-mixed-norm (LMMN) algorithms. For all statements below, we consider a zero mean random variable d with realizations {d(0), d(l),…}, and a zero-mean random row vector u with realizations {u₀, u₁,…}. Moreover, μ is a positive step-size (usually small).

Algorithm 12.1 (Sign-error LMS algorithm) The weight vector w° that solves (in terms of the mean of the l₁ norm — cf. Probs. lll.14–lll.15 and III.29):

can be approximated iteratively via the recursion

In the above statement, ...