O'Reilly logo

Spark in Action by Petar Zečević Marko Bonaći

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 4. The Spark API in depth

This chapter covers

  • Working with key-value pairs
  • Data partitioning and shuffling
  • Grouping, sorting, and joining data
  • Using accumulators and broadcast variables

The previous two chapters explained RDDs and how to manipulate them with basic actions and transformations. You’ve seen how to run Spark programs from Spark REPL and how to submit standalone applications to Spark.

In this chapter, you’ll delve further into the Spark Core API and become acquainted with a large number of Spark API functions. But don’t faint just yet! We’ll be gentle, go slowly, and take you safely through these complicated and comprehensive, but necessary, topics.

You’ll also learn how to use RDDs of key-value pairs called pair ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required