Mastering Predictive Analytics with scikit-learn and TensorFlow

Exhaustive grid search

One of the most important and generally-used methods for performing hyperparameter tuning is called the exhaustive grid search. This is a brute-force approach because it tries all of the combinations of hyperparameters from a grid of parameter values. Then, for each combination of hyperparameters, the model is evaluated using k-fold cross-validation and any other specified metrics. So the combination that gives us the best metric is the one that is returned by the object that we will use in scikit-learn.

Let's take an example of a hyperparameter grid. Here, we try three different values, such as 10, 30, and 50, for the n_estimators hyperparameter. We will try two options, such as auto and square root, for max_features ...