Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners

© Ekaba Bisong 2019

E. . BisongBuilding Machine Learning and Deep Learning Models on Google Cloud Platformhttps://doi.org/10.1007/978-1-4842-4470-8_33

33. More on Optimization Techniques

Ekaba Bisong¹ 

(1)

OTTAWA, ON, Canada

 

In this chapter, we’ll go over some other optimization techniques for improving the ability of a neural network to learn complex patterns in a dataset.

Momentum

Momentum is a technique for improving the convergence speed of stochastic gradient descent (SGD) optimization. Remember that stochastic gradient works by learning the direction of steepest descent by evaluating a training example at each time step to optimize the weights of the network. Momentum improves on this by calculating the average of previous gradients in a process ...