Hands-On Big Data Analytics with PySpark

Loading Spark from file

Once we have the resulting graph, we can pass the spark and path parameters to our GraphBuilder.loadFromFile() method, and at this point, we'll have a graph that is a construct graph of Graph [Int, Int], as follows:

  val graph = GraphBuilder.loadFromFile(spark, path)

To iterate and validate that our graph was properly loaded, we will use triplets from graph, which are a pair of vertex to vertex and also an edge between those vertices. We will see that the structure of the graph was loaded properly:

    //then    graph.triplets.foreach(println(_))

At the end, we are asserting that we get 4 triplets (as shown earlier in the Creating the loader component section, we have four definitions from the graph.g file):

 assert(graph.triplets.count() ...