Machine Learning Foundations, Volume 1: Supervised Learning

Chapter 6. K-Nearest Neighbors

The k-nearest neighbors algorithm (KNN) is a simple yet powerful machine learning technique. It classifies a new instance by assigning it the most frequent label among its k closest neighbors in the training set. Unlike many learning algorithms that build a model during the training phase, KNN is a lazy learner that defers most of the computation until prediction time. In addition, it is a non-parametric and instance-based algorithm, meaning it makes no assumptions about the underlying data distribution and relies directly on the stored training instances for predictions.

KNN is highly adaptable to various datasets. By leveraging local neighborhoods of data points for prediction, it can form decision boundaries ...