Artificial Intelligence for Big Data

Learning rate

In the Gradient-based learning section of this chapter, we established the equations for weight and bias updates for the deep neural network as follows:

In these equations, the learning rate is denoted by . The learning rate for the gradient descent algorithm defines the size of the step that algorithm takes with each training set instance. If the learning rate is too high, the average loss across the gradient descent steps will ...