For random forests, I will highlight two hyperparameters. One is the maximum tree depth of a single tree. This would be a situation where you want even more shallow trees, as opposed to when you had one decision tree making all the decisions, where you might want a bunch of very shallow trees when training a random forest. The other hyperparameter novel to the random forest is the number of trees to grow.
Here, we are going to be optimizing multiple hyperparameters. When we do this, we have some additional considerations regarding how exactly we're going to optimize them. We have a few ways we could possibly proceed, as follows:
- We could use cross-validation to see which combinations of hyperparameters perform ...
