Gradient Tree Boosting is a technique that allows you to build a tree ensemble step by step (the method is also known as forward stage-wise additive modeling), with the goal of minimizing a target loss function. The generic output of the ensemble can be represented as follows:
Here, cj(x) is a function representing a weak learner (in this particular case, it's always a Decision Tree that can be modeled as a single parametrized function, f(•), where the vector, θi, groups all the splitting tuples of the ith tree). The algorithm is based on the concept of adding a new Decision Tree at each step to minimize a global cost ...
