Let's consider the example of StatePopulation RDD generating a pairRDD of <State, <Year, Population>>.
groupByKey as seen in the preceding section will do HashPartitioning of the PairRDD by generating a hashcode of the keys and then shuffling the data to collect the values for each key in the same partition. This obviously results in too much shuffling.
reduceByKey improves upon groupByKey using a local combiner logic to minimize the data sent in a shuffle phase. The result will be the same as groupByKey, but will be much more performant.
aggregateByKey is very similar to reduceByKey in how it works but with one big difference, which makes it the most powerful one among ...
