O'Reilly logo

Fast Data Processing with Spark 2 - Third Edition by Krishna Sankar

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Building graphs

Now that we have our data, the next step is to create a graph. Let's fire up the Spark Shell. Run it from the directory where you have installed Spark:

Building graphs

We create the graph in two steps: first we create an RDD list of the vertices and edges, then we create RDDs and eventually the graph. You don't need to type the code; the graphx-0x.scala files have the programs to create a graph and do the rest of the API stuff.

First we create the lists:

Building graphs

Let's quickly run through the code:

case class Person(name:String,age:Int) val defaultPerson = ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required