Python Machine Learning

CHAPTER 9Supervised Learning—Classification Using K‐Nearest Neighbors (KNN)

What Is K‐Nearest Neighbors?

Up until this point, we have discussed three supervised learning algorithms: linear regression, logistics regression, and support vector machines. In this chapter, we will dive into another supervised machine learning algorithm known as K‐Nearest Neighbors (KNN).

KNN is a relatively simple algorithm compared to the other algorithms that we have discussed in previous chapters. It works by comparing the query instance's distance to the other training samples and selecting the K‐nearest neighbors (hence its name). It then takes the majority of these K‐neighbor classes to be the prediction of the query instance.

Figure 9.1 sums this up nicely. When k = 3, the closest three neighbors of the circle are the two squares and the one triangle. Based on the simple rule of majority, the circle is classified as a square. If k = 5, then the closest five neighbors are the two squares and the three triangles. Hence, the circle is classified as a triangle.

Figure 9.1: The classification of a point depends on the majority of its neighbors

TIP

KNN is also sometimes used for regression in addition to classification. For example, it can be used to calculate the average of the numerical target of the K‐nearest neighbors. For this chapter, however, we are focusing solely on its more common use ...