Modern Scala Projects

Creating a new data pipeline with hyperparameters

The following steps need to be performed to create a new data pipeline:

1. Create a new pipeline with the following three stages:

val soPipeline = new Pipeline().setStages(Array[PipelineStage](labelIndexer) ++ Array[PipelineStage](randomForestClassifier)) ++ Array[PipelineStage](stringIndexer)]

2. Create a hyperparameter called NumTrees as follows:

//Lets set the hyper parameter NumTreesval rfNum_Trees = randomForestClassifier.setNumTrees(15)println("Hyper Parameter num_trees is: " + rfNum_Trees.numTrees)

3. Create a hyperparameter tree called MaxDepth and set it to 2 as follows:

//set this default parameter in the classifier's embedded param mapval rfMax_Depth = rfNum_Trees.setMaxDepth(2)println ...