Modern Scala Projects

Step 3 – splitting the RDD around the newline character in the dataset

Invoke a flatMap operation over the result1 RDD, and split each partition around its "\n" (end of the line) character as follows:

scala> val result2 = result1.flatMap{ partition => partition.split("\n").toList }result2: org.apache.spark.rdd.RDD[String] = MapPartitionsRDD[2] at flatMap at <console>:27

Each partition is after a string. In the next step, we will transform the result2 RDD.