How it works...

We created a list of edges and vertices for a graph and then proceeded to create the object and set the parameters:

new PowerIterationClustering().setK(3).setMaxIterations(15)

The next step was the model of training data:

val model = pic.run(trainingData)

The clusters were then outputted for inspection. The code near the end prints out the model assignment data in a collection for each cluster using Spark transformation operators.

At the core PIC (Power Iteration Clustering) is an eigenvalue class algorithm which avoids matrix decomposition by producing an Eigen Value plus an Eigen Vector to satisfy Av = λv. Because PIC avoids the decomposition of the matrix A, it is suitable when the input matrix A (describing a graph in ...

Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.