Chapter 4. The Spark API in depth
This chapter covers
- Working with key-value pairs
- Data partitioning and shuffling
- Grouping, sorting, and joining data
- Using accumulators and broadcast variables
The previous two chapters explained RDDs and how to manipulate them with basic actions and transformations. You’ve seen how to run Spark programs from Spark REPL and how to submit standalone applications to Spark.
In this chapter, you’ll delve further into the Spark Core API and become acquainted with a large number of Spark API functions. But don’t faint just yet! We’ll be gentle, go slowly, and take you safely through these complicated and comprehensive, but necessary, topics.
You’ll also learn how to use RDDs of key-value pairs called pair ...
Get Spark in Action now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
