Frank Kane's Taming Big Data with Apache Spark and Python

Map versus flatmap

For the next few sections in this book, we'll look at your standard "count the words in a text file" sample that you see in a lot of these sorts of books, but we're going to do a little bit of a twist. We'll work our way up from a really simple implementation of counting the words, and keep adding more and more stuff to make that even better as we go along. So, to start off with, we need to review the concept of map versus flatMap. We talked about this a little bit in the past, but in our examples so far we've just been using map or mapValues to transform our RDDs.