Mastering Machine Learning with scikit-learn - Second Edition

From Decision Trees to Random Forests and Other Ensemble Methods

An ensemble is a combination of estimators that performs better than each of its components. In this chapter, we will introduce three methods of creating ensembles: bagging, boosting, and stacking. First, we will apply bagging to the decision trees introduced in the previous chapter to create a powerful ensemble called random forest. Then we will introduce boosting and the popular AdaBoost algorithm. Finally, we will use stacking to create ensembles from heterogeneous base estimators.