Hyperparameter Tuning with Python

Chapter 3: Exploring Exhaustive Search

Hyperparameter tuning doesn't always correspond to fancy and complex search algorithms. In fact, a simple for loop or manual search based on the developer's instinct can also be utilized to achieve the goal of hyperparameter tuning, which is to get the maximum evaluation score on the validation score without causing an overfitting issue.

In this chapter, we'll discuss the first out of four groups of hyperparameter tuning, called an exhaustive search. This is the most widely used and most straightforward hyperparameter-tuning group in practice. As explained by its name, hyperparameter-tuning methods that belong to this group work by exhaustively searching through the hyperparameter space. Except for one ...