Hands-On Artificial Intelligence for Beginners

Hyperparameter optimization

Aside from protecting against overfitting, we can optimize models by searching for the best combination of model hyperparameters. Hyperparameters are configuration variables that tell the model what methods to use, as opposed to model parameters which are learned during training - we'll learn more about these in upcoming chapter. They are programmatically added to a model, and are present in all modeling packages in Python. In the random forest model that we built precedingly, for instance, n_estimators is a hyperparameter that tells the model how many trees to build. The process of searching for the combination of hyperparameters that leads to the best model performance is called hyperparameter tuning. 

In Python, ...