Machine Learning Foundations, Volume 1: Supervised Learning

Chapter 6

K-Nearest Neighbors

The k-nearest neighbors algorithm (KNN) is a simple yet powerful machine learning technique. It classifies a new instance by assigning it the most frequent label among its k closest neighbors in the training set. Unlike many algorithms that build an explicit model during training, KNN is a lazy learner: it stores the training data and defers most computation until prediction time. It is also a non-parametric, instance-based algorithm, meaning it makes no assumptions about the underlying data distribution and relies directly on the stored training instances for predictions.

Despite its simplicity, KNN can perform remarkably well and sometimes even outperform more sophisticated models in domains where local structure ...