Building Applications on Mesos

Chapter 6. Advanced Topics in Mesos

Although you now know the ropes of building applications on Mesos, you’ve probably got even more questions than when you started:

• What is the internal architecture of Mesos?

• How does Mesos handle failures?

• How can I use Docker with Mesos?

In this chapter, we’ll learn more about all these things, so that you can build even more sophisticated, reliable systems.

Libprocess and the Actor Model

At a high level, Mesos is a hybrid strongly consistent/eventually consistent system, based on the message passing actor model, written in C++.

The actor model is a paradigm for programming concurrent and distributed systems. In the actor model, the developer implements processes or actors, which are single-threaded modules of code that process messages one at a time. While processing a message, an actor can send messages to other actors or create new actors. Actors can only send messages to those whose process identifiers (PIDs) they know. When an actor spawns another actor, it learns the PID of the newly created actor. In order to get more PIDs, actors must send the PIDs to one another in messages.

The actor model framework that Mesos uses is called libprocess, and it was written by Ben Hindman at the same time he wrote Mesos.¹ Libprocess is a C++ API that provides actor model semantics and uses HTTP to communicate. Since everything speaks HTTP, it’s easy to interface with libprocess.

Libprocess messages are just protobufs over HTTP, and can benefit ...