Frank Kane's Taming Big Data with Apache Spark and Python

Step 2 - Sort the new RDD

What we have at this point is an RDD that contains words and the number of times they appear. We want to sort by the number of times each word appears. So before we sort, we need to flip that around and map things so that our keys become values and our values become keys. We start off with the keys being the words and the values being the number of times that the word occurred, then we flip that around in this mapper shown here. We now have the number of times the word occurred followed by the word itself:

wordCountsSorted = wordCounts.map(lambda x: (x[1], x[0])).sortByKey() 

Then, we can chain together a call to sortByKey in order to sort the final RDD that we store in wordCountSorted by the number of times it ...