Hands-On Recommendation Systems with Python

Boosting

The bagging and the random forest models train baseline models that are completely independent of each other. Therefore, they do not learn from the mistakes that each learner has made. This is where boosting comes into play.

Like random forests, boosting models build a baseline model using a subset of samples and features. However, while building the next learners, the boosting model tries to rectify the mistakes that the previous learners made. Different boosting algorithms do this in different ways.

For example, the original boosting algorithm simply added 50% of the misclassified samples to the second learner, and all the samples that the first two learners disagree upon to build the third and final learner. This ensemble of three ...