January 2019
Beginner to intermediate
154 pages
4h 31m
English
The foreach() function applies a function to each element of the RDD. The following example concatenates the string Mr. to each element using foreach():
//Scalaspark.sparkContext.parallelize(Array("Smith","John","Brown","Dave")).foreach{ x => println("Mr. "+x) }
If you run the previous example in local mode, you will see the output. But, in the case of cluster mode, you won't be able to see the results, because foreach() performs the given function inside the executors and does not return any data to the driver.
This is mainly used to work with accumulators. We shall see this in more detail in Chapter 5, Spark Architecture and Application Execution Flow.
You can find more transformations and actions at https://spark.apache.org/docs/2.3.0/rdd-programming-guide.html#transformations. ...
Read now
Unlock full access