Machine Learning with Spark - Second Edition

Iterations

As we saw when evaluating our classification models, we generally expect that a model trained with SGD will achieve better performance as the number of iterations increases, although the increase in performance will slow down as the number of iterations goes above some minimum number. Note that here, we will set the step size to 0.01 to better illustrate the impact at higher iteration numbers.

We implemented the same in Scala with different values of iterations, as follows:

val data = LinearRegressionUtil.getTrainTestData() val train_data = data._1 val test_data = data._2 val iterations = 10 //LinearRegressionCrossValidationStep$ //params = [1, 5, 10, 20, 50, 100, 200] val iterations_param = Array(1, 5, 10, 20, 50, 100, 200)  ...