Apache Spark 2.x Cookbook

How to do it...

1. Start the Spark shell:

         $ spark-shell

2. Perform the required imports:

        scala> import org.apache.spark.ml.classification.        {RandomForestClassificationModel,RandomForestClassifier}        scala> import         org.apache.spark.ml.evaluation.MulticlassClassificationEvaluator

3. Load and parse the data:

        scala> val data =         spark.read.format("libsvm").load("s3a://sparkcookbook/rf")

4. Split the data into training and test datasets:

        scala> val Array(training, test) = data.randomSplit(Array(0.7, 0.3))

5. Create a classification as a tree strategy (random forest also supports regression):

        scala> val rf = new RandomForestClassifier().setNumTrees(3)

6. Train the model:

        scala> val model = rf.fit(training)

7. Evaluate the model on test instances ...