Hands-On Machine Learning for Algorithmic Trading

Regularization

All libraries implement the regularization strategies for base learners, such as minimum values for the number of samples or the minimum information gain required for splits and leaf nodes.

They also support regularization at the ensemble level using shrinkage via a learning rate that constrains the contribution of new trees. It is also possible to implement an adaptive learning rate via callback functions that lower the learning rate as the training progresses, as has been successfully used in the context of neural networks. Furthermore, the gradient boosting loss function can be regularized using L1 or L2, regularization similar to the Ridge and Lasso linear regression models by modifying Ω(h_m) or by increasing the penalty ...