Machine Learning Foundations, Volume 1: Supervised Learning

Chapter 9

Ensemble Methods

Ensemble methods are a powerful and widely used class of machine learning techniques that combine multiple base models to form a more accurate and robust predictor than any individual model alone.

The core idea behind ensemble learning is that aggregating the predictions of diverse models—each of which may capture different patterns or make different errors—can lead to better generalization. Ensembles also tend to reduce variance by averaging the predictions of multiple simpler models, rather than relying on a single, complex model that might overfit the training data. Furthermore, ensembles can effectively expand the hypothesis space, enabling them to model more complex patterns than any individual model could represent ...