Hands-On Machine Learning for Algorithmic Trading

Overfitting and regularization

Decision trees have a strong tendency to overfit, especially when a dataset has a large number of features relative to the number of samples. As discussed in previous chapters, overfitting increases the prediction error because the model does not only learn the signal contained in the training data, but also the noise.

There are several ways to address the risk of overfitting:

• Dimensionality reduction (Chapter 12, Unsupervised Learning) improves the feature-to-sample ratio by representing the existing features with fewer, more informative, and less noisy features.
• Ensemble models, such as random forests, combine multiple trees while randomizing the tree construction, as we will see in the second part of this ...