Deep Learning: Practical Neural Networks with Java

Advanced modeling with ensembles

In the previous section, we implemented an orientation baseline, so let's focus on heavy machinery. We will follow the approach taken by the KDD Cup 2009 winning solution developed by the IBM Research team (Niculescu-Mizil and others, 2009).

Their strategy to address the challenge was using the Ensemble Selection algorithm (Caruana and Niculescu-Mizil, 2004). This is an ensemble method, which means it constructs a series of models and combines their output in a specific way to provide the final classification. It has several desirable properties as shown in the following list that make it a good fit for this challenge:

• It was proven to be robust, yielding excellent performance
• It can be optimized for a specific performance ...