Hands-On Data Science and Python Machine Learning

Exploring decision trees code

So, we are just looking at a raw Python script file now, without any of the usual embellishment of the IPython notebook stuff. let's walk through what's going on in the script.

We'll go through it slowly, because this is your first Spark script that you've seen in this book.

First, we're going to import, from pyspark.mllib, the bits that we need from the machine learning library for Spark.

from pyspark.mllib.regression import LabeledPoint 
from pyspark.mllib.tree import DecisionTree 

We need the LabeledPoint class, which is a data type required by the DecisionTree class, and the DecisionTree class itself, imported ...