O'Reilly logo

Machine Learning by Sergios Theodoridis

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 5

Stochastic Gradient Descent

The LMS Algorithm and its Family

Abstract

The focus of this chapter is to introduce the stochastic gradient descent family of online/adaptive algorithms in the framework of the squared error loss function. The gradient descent approach to optimization is presented and the stochastic approximation method is discussed. Then, the LMS algorithm and its offsprings, such as the APA and the NLMS are introduced. Finally, distributed learning is discussed with an emphasis to distributed versions of the LMS.

Keywords

Affine projection algorithm

Distributed learning

Diffusion LMS

Gradient descent method

Least-mean-squares LMS adaptive algorithm

Method of stochastic approximation

Robbins-Monro algorithm

Steepest ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required