Our algorithm for finding these connected components relies on the threshold parameter, which dictates whether edges are added to the graph or not. In turn, this directly dictates how many connected components we discover and how big they are. From here, we probably want to settle on some notion of which is the best threshold to use. This is a very subjective problem, and there is no definitive answer. This is a major problem with any cluster analysis task.
We can, however, determine what we think a good solution should look like and define a metric based on that idea. As a general rule, we usually want a solution where:
- Samples in the same cluster (connected components) are highly similar to each other
- Samples in different ...
