Building Applications on Mesos

Chapter 5. Building a Mesos Executor

Now you’ve seen how to build a scheduler for Mesos. However, there are some things that can’t be done with the scheduler API alone. For instance, perhaps you’d like to run several tasks in the same container. Maybe your application has lifecycle messages for the scheduler, reporting on its progress or application-specific statistics. Or perhaps you’d like to layer additional functionality into the executed task. In order to accomplish these things, you need to write a custom Mesos executor.

This is what we’ll learn how to do in this chapter. Initially, we’ll simply provide the functionality of the built-in CommandExecutor, which we learned about at the end of the last chapter. Then, we’ll add support for heartbeats to enable faster failure detection. Finally, we’ll discuss potential designs for progress reporting, improved logging, and running multiple tasks in the same container.

The Executor

We’ve already learned what a scheduler is: it’s the component that interacts with the Mesos masters and the framework’s clients, manages the running tasks, and handles failovers. But what is an executor? An executor has three responsibilities:

• Executing tasks as requested by the scheduler

• Keeping the scheduler informed of the status of those tasks

• Handling other requests from the scheduler

You Probably Don’t Want to Do This

Writing executors is very tedious. Unfortunately, the only way to test whether the communication between your scheduler ...