Hands-On Data Analysis with Pandas

Ensemble methods

Ensemble methods combine many models (often weak ones) to create a stronger one that will either minimize average error between observed and predicted values (the bias), or improve how well it generalizes to unseen data (minimize the variance). We have to strike a balance between complex models that may increase variance, as they tend to overfit, and simple models that may have high bias, as these tend to underfit. This is called the bias-variance trade-off, which is illustrated in the following subplots:

Ensemble methods can be broken down into three categories: boosting, bagging, and stacking. Boosting trains many weak ...