Chapter 1: Gradient descent-type methods
Background and simple unified convergence analysis
Quoc Tran-Dinha; Marten van Dijkb,c aDepartment of Statistics and Operations Research, The University of North Carolina at Chapel Hill, Chapel Hill, NC, United StatesbCWI, Amsterdam, NetherlandscVU University Amsterdam, Amsterdam, Netherlands
Abstract
In this book chapter, we briefly describe the main components that constitute the gradient descent method and its accelerated and stochastic variants. We aim at explaining these components from a mathematical point of view, including theoretical and practical aspects, but at an elementary level. We will focus on basic variants of the gradient descent method and then extend our view to recent variants, especially ...
Get Federated Learning now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
