Data Smart: Using Data Science to Transform Information into Insight

5

Cluster Analysis Part II: Network Graphs and Community Detection

This chapter continues the discussion on cluster identification and analysis using the wholesale wine dataset from Chapter 2. Although it's perfectly fine to jump around in this book, in this case I recommend at least skimming Chapter 2 before reading this chapter, because I don't repeat the data preparation steps, and you're going to be using cosine similarity, which was discussed at the end of Chapter 2.

Also, the techniques used here rely on the “Big M” constraint optimization techniques introduced in Chapter 4, so some familiarity with that will be helpful.

This chapter continues addressing the problem of detecting interesting groups of customers based on their purchases, but it approaches the problem from a fundamentally different direction.

Rather than thinking about customers huddling around flags planted on the dance floor to assign them to groups, as you did with k-means clustering (Chapter 2), you're going to look at your customers in a more relational way. Customers buy similar things, and in that way, they're related to each other. Some are more “friendly” than others, in that they're interested in the same stuff. So by thinking about how related or not related each customer is to the others, you can identify communities of customers without needing to plant a set number of flags in the data that get moved around until people feel at home.

The key concept that allows you to approach customer clustering ...