We presented a very basic example of stacking. Some possible extensions include:
- Adding more layers to the stacking ensemble
- Experimenting with different meta learners
- Using more diversified models (for example, SVMs)
- Tuning the hyperparameters of the base classifiers
The ensemble module of scikit-learn also contains a VotingClassifier, which can aggregate the predictions of multiple classifiers. VotingClassifier uses one of the two available voting schemes. The first one is hard, and it is simply the majority vote. The soft voting scheme uses the argmax of the sums of the predicted probabilities to predict the class label.
There are also other libraries providing stacking functionalities. Some of them are listed here: