Machine Learning Algorithms - Second Edition

Homogeneity

An important requirement for a clustering algorithm (given the ground truth) is that each cluster should only contain samples belonging to a single class. In Chapter 2, Important Elements in Machine Learning, we have defined the concepts of entropy H(X) and conditional entropy H(X|Y), which measures the uncertainty of X given the knowledge of Y. Therefore, if the class set is denoted as C and the cluster set as K, H(C|K) is a measure of the uncertainty in determining the right class after having clustered the dataset. To have a homogeneity score, it's necessary to normalize this value considering the initial entropy of the class set, H(C):

If we define the function n(i_true, j_pred), which corresponds to the number of samples ...