Apache Spark Deep Learning Cookbook

How to do it...

This section walks through the steps to fine-tune the model.

1. Define a new logistic regression model with additional parameters for regParam and elasticNetParam as seen in the following script:

logregFT = LogisticRegression( regParam=0.05,  elasticNetParam=0.3, maxIter=15,labelCol = "label", featuresCol="features")

2. Create a new pipeline configured for the newly created model using the following script:

pipelineFT = Pipeline(stages=[vectorizer, logregFT])

3. Fit the pipeline to the trained dataset, trainDF, using the following script:

pipeline_model_FT = pipelineFT.fit(trainDF)

4. Apply the model transformation to the test dataset, testDF, to be able to compare actual versus predicted scores using the following script:

predictDF_FT ...