Frank Kane's Taming Big Data with Apache Spark and Python

Flatmap ()

In contrast, flatMap has the ability to blow out an RDD into multiple entries. So you will end up with an RDD that has more elements than you started with when you use flatMap. As an example, let's see how that works. Look at the following figure. Let's say that we load up that same text file in the lines RDD and call flatMap on it. We have our mapping function as the split function:

What this does is that it splits up every line into individual words based on whitespaces, and outputs multiple results, one for each word. So x.split returns a list of words, and each item returned in that list from the mapper function will become ...