Machine Learning with scikit-learn Quick Start Guide

The gradient boosted tree

In this section, you will learn how the gradient boosted tree is used for regression, and how you can implement this using scikit-learn.

In the AdaBoost classifier that you learned about earlier in this chapter, weights are added to the examples that the classifier predicted in correctly. In the gradient boosted tree, however, instead of weights, the residual errors are used as labels in each tree in order to make future predictions. This concept is illustrated for you in the following diagram:

Here is what occurs in the preceding diagram:

1. The first decision tree is trained with the data that you have, and the target ...