Adaptive Filters

CHAPTER 14 RLS Algorithm

A second example of an algorithm that employs a more sophisticated approximation for R_u is the Recursive-Least-Squares (RLS) algorithm. Although this algorithm can be motivated and derived as the exact solution to a well-defined estimation problem with a least-squares cost function, as we shall show in detail later in the book (see, e.g., Sec. 30.6), we shall motivate it here as simply a stochastic-gradient method. In this way, readers can get an early introduction to this important adaptive algorithm.

14.1 INSTANTANEOUS APPROXIMATION

Just like ∈-NLMS and ∈–APA, we again start from the regularized Newton’s recursion (10.8), namely,

(14.1)

and replace

by the instantaneous approximation

Now, however, we replace R_u by a better estimate for it, which we choose as the exponentially weighted sample average

for some scalar 0<<λ≤1. Assume first that λ = 1. Then the above expression for amounts to averaging all past regressors up to time i, namely,

Choosing a value ...