Natural Language Processing with Python Quick Start Guide

Removing correlated classifiers

Let's look at this in action by taking three simple models as an example. As you can see, the ground truth is all 1s:

1111111100 = 80% accuracy 1111111100 = 80% accuracy 1011111100 = 70% accuracy

These models are highly correlated in their predictions. When we take a majority vote, we see no improvement:

1111111100 = 80% accuracy

Now, let's compare that to the following three lower-performing but highly uncorrelated models:

1111111100 = 80% accuracy 0111011101 = 70% accuracy 1000101111 = 60% accuracy

When we ensemble this with a majority vote, we get the following result:

1111111101 = 90% accuracy

Here, we see a much higher rate of improvement than in any of our individual models. Low correlation between ...