Apache Spark 2.x Cookbook

There's more...

Let's apply the decision tree to the diabetes dataset we worked on in the previous recipe:

1. Start the Spark shell or the Databricks Cloud shell and do the necessary imports:

        $ spark-shell        scala> import         org.apache.spark.ml.classification.DecisionTreeClassifier        scala> import         org.apache.spark.ml.evaluation.BinaryClassificationEvaluator

2. Read the diabetes data as a DataFrame:

        scala> val data =         spark.read.format("libsvm").option("inferschema","true")        .load("s3a://sparkcookbook/medicaldata/diabetes.libsvm")

3. Split it into training and test datasets:

        scala> val Array(trainingData, testData) =         data.randomSplit(Array(0.7, 0.3))

4. Initialize the decision tree classifier:

 scala> val dt = new DecisionTreeClassifier() ...