Mastering Machine Learning with Spark 2.x

Spark random forest model

Next, we will move on to our random forest algorithm, which, as you will recall from the previous chapters, is an ensemble of various decision trees whereby we perform a grid search again alternating between various depths and other hyper-parameters, which will be familiar:

import org.apache.spark.ml.classification.{RandomForestClassifier, RandomForestClassificationModel}val rfModelPath= s"$MODELS_DIR/rfModel"val rfModel= {  val rfGridSearch = for (    rfNumTrees<- Array(10, 15);    rfImpurity<- Array("entropy", "gini");    rfDepth<- Array(3, 5))    yield {      println( s"Training random forest: numTrees: $rfNumTrees,               impurity $rfImpurity, depth: $rfDepth")     val rfModel = new RandomForestClassifier() .setFeaturesCol(idf.getOutputCol) ...