Apache Spark 2.x Cookbook

Getting ready

We will build a small graph here for the clusters we know and use connected components to segregate them. Let's look at the following data:

Follower	Followee
John	Pat
Pat	Dave
Gary	Chris
Chris	Bill

The preceding data is a simple one, with six vertices and two clusters. Let's put this data in the form of two files: nodes.csv and edges.csv.

The following is the content of nodes.csv:

1,John 2,Pat 3,Dave 4,Gary 5,Chris 6,Bill 

The following is the content of edges.csv:

1,2,follows 2,3,follows 4,5,follows 5,6,follows 

We should expect a connected component algorithm to identify two clusters, the first one is identified ...