Learning Data Mining with Python - Second Edition

Nearest neighbors

The Nearest neighbors algorithm is our new sample. We take the most similar samples and predict the same class that most of these nearby samples have. This vote is often simply a simple count,although more complicated methods do exist such as weighted voting.

As an example in the below diagram, we wish to predict the class of the triangle, based on which class it is more like (represented here by having similar objects closer together). We seek the three nearest neighbors, which are the two diamonds and one square within the drawn circle. There are more diamonds than circles, and the predicted class for the triangle is, therefore, a diamond:

Nearest neighbors are used for nearly any dataset - however, it can be computationally ...