Mastering Machine Learning with Spark 2.x

Spark decision tree model

First, let's start with a simple decision tree and perform a grid search over a few of the hyper-parameters. We will follow the code from Chapter 2, Detecting Dark Matter: The Higgs-Boson Particle to build our models that are trained to maximize the AUC statistic. However, instead of using models from the MLlib library, we will adopt models from the Spark ML package. The motivation of using the ML package will be clearer later when we will need to compose the models into a form of pipeline. Nevertheless, in the following code, we will use DecisionTreeClassifier, which we fit to trainData, generate prediction for testData, and evaluate the model's AUC performance with the help of BinaryClassificationEvaluato:

import ...