Machine Learning Algorithms

Gradient tree boosting

Gradient tree boosting is a technique that allows you to build a tree ensemble step by step with the goal of minimizing a target loss function. The generic output of the ensemble can be represented as:

 

Here, f_i(x) is a function representing a weak learner. The algorithm is based on the concept of adding a new decision tree at each step so as to minimize the global loss function using the steepest gradient descent method (see https://en.wikipedia.org/wiki/Method_of_steepest_descent, for further information):

After introducing ...