Mastering Machine Learning with Spark 2.x

Spark GBM model

Finally, we will move on to our gradient boosting machine (GBM), which will be the final model in our ensemble of models. Note that in the previous chapters, we used H2O's version of GBM, but now, we will stick with Spark and use Spark's implementation of GBM as follows:

import org.apache.spark.ml.classification.{GBTClassifier, GBTClassificationModel}val gbmModelPath= s"$MODELS_DIR/gbmModel"val gbmModel= {  val model = new GBTClassifier()      .setFeaturesCol(idf.getOutputCol)      .setLabelCol("label")      .setMaxIter(20)      .setMaxDepth(6)      .setCacheNodeIds(true)      .fit(trainData)  val gbmPrediction = model.transform(testData)  gbmPrediction.show()  val gbmAUC = new BinaryClassificationEvaluator()      .setLabelCol("label") .setRawPredictionCol(model.getPredictionCol) ...