Mastering Machine Learning with Spark 2.x

Aggregating messages

First, we discuss the aggregateMessages method that GraphX graphs come with. The basic idea is to pass messages along edges in parallel across the whole graph, aggregate these messages suitably and store the result for further processing. Let's have a closer look at how aggregateMessages is defined:

def aggregateMessages[Msg: ClassTag](  sendMsg: EdgeContext[VD, ED, Msg] => Unit,  mergeMsg: (Msg, Msg) => Msg,  tripletFields: TripletFields = TripletFields.All): VertexRDD[Msg]

As you can see, to implement an aggregateMessages algorithm we need to specify a message type Msg and provide three functions, which we will explain next. You may notice that there are two additional types that we haven't encountered before, namely ...