Advances in Financial Machine Learning

CHAPTER 9 Hyper-Parameter Tuning with Cross-Validation

9.1 Motivation

Hyper-parameter tuning is an essential step in fitting an ML algorithm. When this is not done properly, the algorithm is likely to overfit, and live performance will disappoint. The ML literature places special attention on cross-validating any tuned hyper-parameter. As we have seen in Chapter 7, cross-validation (CV) in finance is an especially difficult problem, where solutions from other fields are likely to fail. In this chapter we will discuss how to tune hyper-parameters using the purged k-fold CV method. The references section lists studies that propose alternative methods that may be useful in specific problems.

9.2 Grid Search Cross-Validation

Grid search cross-validation conducts an exhaustive search for the combination of parameters that maximizes the CV performance, according to some user-defined score function. When we do not know much about the underlying structure of the data, this is a reasonable first approach. Scikit-learn has implemented this logic in the function GridSearchCV, which accepts a CV generator as an argument. For the reasons explained in Chapter 7, we need to pass our PurgedKFold class (Snippet 7.3) in order to prevent that GridSearchCV overfits the ML estimator to leaked information.

Snippet 9.1 Grid search with purged k-fold cross-validation

Snippet 9.1 lists function ...