Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners

© Ekaba Bisong 2019

E. . BisongBuilding Machine Learning and Deep Learning Models on Google Cloud Platformhttps://doi.org/10.1007/978-1-4842-4470-8_16

16. Optimization for Machine Learning: Gradient Descent

Ekaba Bisong¹ 

(1)

OTTAWA, ON, Canada

 

Gradient descent is an optimization algorithm that is used to minimize the cost function of a machine learning algorithm. Gradient descent is called an iterative optimization algorithm because, in a stepwise looping fashion, it tries to find an approximate solution by basing the next step off its present step until a terminating condition is reached that ends the loop.

Take the following convex function in Figure 16-1 as a visual of gradient descent finding the minimum point of a function space.

Figure 16-1