O'Reilly logo

Machine Learning with Spark - Second Edition by Nick Pentreath, Manpreet Singh Ghotra, Rajdeep Dua

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Step size

In SGD, the step size parameter controls how far in the direction of the steepest gradient the algorithm takes a step when updating the model weight vector after each training example. A larger step size might speed up convergence, but a step size that is too large might cause problems with convergence, as good solutions are overshot. The learning rate determines the size of the steps we take to reach a (local or global) minimum. In other words, we follow the direction of the slope of the surface created by the objective function downhill until we reach a valley.

We can see the impact of changing the step size here:

val stepResults = Seq(0.001, 0.01, 0.1, 1.0, 10.0).map { param =>  val model = trainWithParams(scaledDataCats, 0.0, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required