O'Reilly logo

Spark in Action by Petar Zečević Marko Bonaći

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 12. Running on YARN and Mesos

This chapter covers

  • YARN architecture
  • YARN resource scheduling
  • Configuring and running Spark on YARN
  • Mesos architecture
  • Mesos resource scheduling
  • Configuring and running Spark on Mesos
  • Running Spark from Docker

We examined a Spark standalone cluster in the previous chapter. Now it’s time to tackle YARN and Mesos, two other cluster managers supported by Spark. They’re both widely used (with YARN still more widespread) and offer similar functionalities, but each has its own specific strengths and weaknesses. Mesos is the only cluster manager supporting fine-grained resource scheduling mode; you can also use Mesos to run Spark tasks in Docker images. In fact, the Spark project was originally started ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required