Hyperparameter Tuning with Python

Chapter 5: Exploring Heuristic Search

Heuristic search is the third out of four groups of hyperparameter tuning methods. The key difference between this group and the other groups is that all the methods that belong to this group work by performing trial and error to achieve the optimal solution. Similar to the acquisition function in Bayesian optimization (see Chapter 4, Exploring Bayesian Optimization), all methods in this group also employ the concept of exploration versus exploitation. Exploration means performing a search in the unexplored space to lower the probability of being stuck in the local optima, while exploitation means performing a search in the local space that is known to have a good chance of containing the optimal solution. ...