Hands-On Machine Learning for Algorithmic Trading

Second-order loss function approximation

The most important algorithmic innovations lower the cost of evaluating the loss function by using approximations that rely on second-order derivatives, resembling Newton's method to find stationary points. As a result, scoring potential splits during greedy tree expansion is faster relative to using the full loss function.

As mentioned previously, a gradient boosting model is trained in an incremental manner with the goal of minimizing the combination of the prediction error and the regularization penalty for the ensemble H_M. Denoting the prediction of the outcome y_i by the ensemble after step m as ŷ_i^(m), l as a differentiable convex loss function that measures the difference between the outcome and ...