Apache Spark Deep Learning Cookbook

How to do it...

This section will walk through the steps of applying our model and evaluating the results.

1. Execute the following script to lump all of the feature variables in the dataframe in a list called features:

features = df.columns[1:]

2. Execute the following to import VectorAssembler and configure the fields that will be assigned to the feature vector by assigning the inputCols and outputCol:

from pyspark.ml.feature import VectorAssemblerfeature_vectors = VectorAssembler(    inputCols = features,    outputCol = "features")

3. Execute the following script to apply VectorAssembler to the dataframe with the transform function:

df = feature_vectors.transform(df)

4. Modify the dataframe to remove all of the columns except for fireIndicator and ...