Machine Learning with Spark - Second Edition

ALS Model Evaluation

From Spark v2.0, we will use org.apache.spark.ml.evaluation.RegressionEvaluator for regression problems. Regression evaluation is a metric to measure how well a fitted model does on held-out test data. Here, we will use Root Mean Squared Error (RMSE), which is just the square root of the MSE metric:

object ALSModeling {   def createALSModel() {     val ratings = FeatureExtraction.getFeatures();     val Array(training, test) = ratings.randomSplit(Array(0.8, 0.2))     println(training.first())     // Build the recommendation model using ALS on the training data     val als = new ALS()       .setMaxIter(5)       .setRegParam(0.01)       .setUserCol("userId")       .setItemCol("movieId")       .setRatingCol("rating")     val model = als.fit(training)  println(model.userFactors.count()) ...