Machine Learning Algorithms - Second Edition

K-means

In the previous section, we discussed an algorithm based on the assumption that the data-generating process can be represented as a weighted sum of multivariate Gaussian distributions. What happens when the covariance matrices are shrunk towards zero? As it's easy to imagine, when Σ_i → 0, the corresponding distribution degenerates to a Dirac's Delta centered on the mean. In other words, the probability will become almost 1 if the sample is extremely close to the mean, and 0 otherwise. In this case, the membership to a cluster becomes binary and it's determined only by the distance between the sample and the mean (the shortest distance will determine the winning cluster).

The K-means algorithm is the natural hard extension of Gaussian ...