Python Data Analysis Cookbook

Stacking and majority voting for multiple models

It is generally believed that two people know more than one person alone. A democracy should work better than a dictatorship. In machine learning, we don't have humans making decisions, but algorithms. When we have multiple classifiers or regressors working together, we speak of ensemble learning.

There are many ensemble learning schemes. The simplest setup does majority voting for classification and averaging for regression. In scikit-learn 0.17, you can use the VotingClassifier class to do majority voting. This classifier lets you emphasize or suppress classifiers with weights.

Stacking takes the outputs of machine learning estimators and then uses those as inputs for another algorithm. You can, ...