Hands-On Genetic Algorithms with Python

Hyperparameter tuning

A common way to search for good combinations of hyperparameters is by using a grid search. Using this method, we choose a subset of values for each hyperparameter that we want to tune. As an example, given the Decision Tree classifier, we can choose the subset of values {2, 5, 10} for the max_depth parameter, while, for the splitter parameter, we choose both possible values – {"best", "random"}. Then, we try out all six possible combinations of these values. For each combination, the classifier is trained and evaluated for a certain performance criterion, for example, accuracy. At the end of the process, we pick the combination of hyperparameter values that yielded the best performance.

The main drawback of the grid ...