O'Reilly logo

Hands-On Machine Learning with C# by Matt R. Cole

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Learning rate parameter

The learning rate is a parameter that determines the speed of learning. More formally, it determines how much we are adjusting the weights of our network with respect to the loss gradient. If it is too low, we travel slower down our slope. Even though we desire to have a low learning rate, it could mean that we'll be taking a long time to reach convergence. The learning rate also affects how quickly our model can converge tolocal minima (best accuracy).

When dealing with neurons, it determines the acquisition time (the amount of time it takes for a response to a new experience to be learned) for neurons with weights being used for training.

Effect of various learning rates on convergence (image credit: cs231n)

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required