O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Apache Mesos Cookbook

Book Description

Over 50 recipes on the core features of Apache Mesos and running big data frameworks in Mesos

About This Book

  • Learn to install and configure Mesos to suit the needs of your organization
  • Follow step-by-step instructions to deploy application frameworks on top of Mesos, saving you many hours of research and trial and error
  • Use this practical guide packed with powerful recipes to implement Mesos and easily integrate it with other application frameworks

Who This Book Is For

This book is for system administrators, engineers, and big data programmers. Basic experience with big data technologies such as Hadoop or Spark would be useful but is not essential. A working knowledge of Apache Mesos is expected.

What You Will Learn

  • Set up Mesos on different operating systems
  • Use the Marathon and Chronos frameworks to manage multiple applications
  • Work with Mesos and Docker
  • Integrate Mesos with Spark and other big data frameworks
  • Use networking features in Mesos for effective communication between containers
  • Configure Mesos for high availability using Zookeeper
  • Secure your Mesos clusters with SASL and Authorization ACLs
  • Solve everyday problems and discover the best practices

In Detail

Apache Mesos is open source cluster sharing and management software. Deploying and managing scalable applications in large-scale clustered environments can be difficult, but Apache Mesos makes it easier with efficient resource isolation and sharing across application frameworks.

The goal of this book is to guide you through the practical implementation of the Mesos core along with a number of Mesos supported frameworks. You will begin by installing Mesos and then learn how to configure clusters and maintain them. You will also see how to deploy a cluster in a production environment with high availability using Zookeeper.

Next, you will get to grips with using Mesos, Marathon, and Docker to build and deploy a PaaS. You will see how to schedule jobs with Chronos. We'll demonstrate how to integrate Mesos with big data frameworks such as Spark, Hadoop, and Storm. Practical solutions backed with clear examples will also show you how to deploy elastic big data jobs.

You will find out how to deploy a scalable continuous integration and delivery system on Mesos with Jenkins. Finally, you will configure and deploy a highly scalable distributed search engine with ElasticSearch.

Throughout the course of this book, you will get to know tips and tricks along with best practices to follow when working with Mesos.

Style and approach

This step-by-step guide is packed with powerful recipes on using Apache Mesos and shows its integration with containers and big data frameworks.

Table of Contents

  1. Preface
    1. What this book covers
    2. What you need for this book
    3. Who this book is for
    4. Sections
      1. Getting ready
      2. How to do it…
      3. How it works…
      4. There's more…
      5. See also
    5. Conventions
    6. Reader feedback
    7. Customer support
      1. Downloading the example code
      2. Downloading the color images of this book
      3. Errata
      4. Piracy
      5. Questions
  2. Getting Started with Apache Mesos
    1. Introduction
    2. Installing Mesos on Ubuntu 16.04 from packages
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    3. Installing Mesos on Ubuntu 14.04 from packages
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    4. Installing Mesos on CentOS 7 and RHEL 7 from packages
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    5. Preparing Ubuntu 16.04 for a Mesos installation from source code
      1. Getting ready
      2. How to do it...
      3. How it works...
    6. Preparing Ubuntu 14.04 for a Mesos installation from source code
      1. Getting ready
      2. How to do it...
      3. How it works...
    7. Preparing OS X (Yosemite and El Capitan) for a Mesos Installation from source code
      1. Getting ready
      2. How to do it...
      3. How it works...
    8. Downloading, building, and installing the Mesos source code
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Configuration tips
        2. Build tips
        3. Installation tips
  3. Implementing High Availability with Apache ZooKeeper
    1. Introduction
    2. Configuring ZooKeeper installed from packages on Ubuntu 14.04, Ubuntu 16.04, CentOS 7, or RHEL 7
      1. Getting ready
      2. How to do it...
        1. Configuring Mesos masters with ZooKeeper
        2. Configuring Mesos agents with ZooKeeper
      3. How it works...
      4. There's more...
    3. Configuring ZooKeeper on Ubuntu 14.04, 16.04, and OS X from source
      1. Getting ready
      2. How to do it...
        1. Configuring Mesos masters with ZooKeeper
        2. Configuring Mesos agents with ZooKeeper
      3. How it works...
      4. There's more...
  4. Running and Maintaining Mesos
    1. Introduction
    2. Logging and debugging
      1. Getting ready
      2. How to do it...
      3. How it works...
    3. Monitoring
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    4. Setting attributes for agents
      1. How to do it...
      2. How it works...
    5. Defining roles and resources
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Dynamic reservations
        2. Persistent volumes
    6. Oversubscription
      1. How to do it...
      2. How it works...
      3. See also
    7. Controlling permissions using access control lists
      1. Getting ready
      2. How to do it...
      3. How it works...
    8. Agent authentication
      1. Getting ready
      2. How to do it...
        1. Master
        2. Agent
        3. Marathon
      3. How it works...
    9. Encryption SSL
      1. Getting ready
      2. How to do it...
      3. How it works...
    10. Upgrading Mesos
      1. How to do it...
      2. How it works...
  5. Understanding the Scheduler API
    1. Introduction
    2. Installing Protobuf
      1. Getting ready
      2. How to do it...
      3. How it works...
    3. Registering frameworks
      1. How to do it...
      2. How it works...
    4. Handling events
      1. How to do it...
      2. How it works...
    5. Declining offers
      1. How to do it...
      2. How it works...
    6. Scheduling tasks
      1. How to do it...
      2. How it works...
    7. Acknowledging task updates
      1. How to do it...
      2. How it works...
    8. Killing tasks
      1. How to do it...
      2. How it works...
    9. State persistence
      1. How to do it...
      2. How it works...
    10. Reconciliation
      1. How to do it...
      2. How it works...
  6. Managing Containers
    1. Introduction
    2. Enabling the Mesos containerizer
      1. Getting ready
      2. How to do it...
      3. How it works...
    3. Enabling POSIX isolators
      1. Getting ready
      2. How to do it....
      3. How it works...
    4. Enabling the POSIX disk isolator
      1. Getting ready
      2. How to do it...
      3. How it works...
    5. Configuring the shared filesystem isolator
      1. Getting ready
      2. How to do it...
      3. How it works...
    6. Configuring cgroup isolators
      1. Getting ready
      2. How to do it...
      3. How it works...
    7. Configuring the port mapping network isolator
      1. Getting ready
      2. How to do it...
      3. How it works...
    8. Configuring Docker image support for the Mesos containerizer
      1. Getting ready
      2. How to do it...
      3. How it works...
    9. Using the Docker containerizer
      1. Getting ready
      2. How to do it...
      3. How it works...
    10. Running an image from a private repository
      1. Getting ready
      2. How to do it...
      3. How it works...
    11. Using container network interface
      1. Getting ready
      2. How to do it...
      3. How it works...
    12. Monitoring containers with Sysdig
      1. Getting ready
      2. How to do it...
      3. How it works...
  7. Deploying PaaS with Marathon
    1. Introduction
    2. Installing and configuring Marathon
      1. Getting ready
      2. How to do it...
      3. How it works...
    3. Monitoring Marathon
      1. Getting ready
      2. How to do it...
      3. How it works...
    4. Launching an application
      1. Getting ready
      2. How to do it...
      3. How it works...
    5. Launching a Docker container
      1. Getting ready
      2. How to do it...
      3. How it works...
    6. Setting up the discovery service
      1. Getting ready
      2. How to do it...
      3. How it works...
    7. Enabling SSL
      1. Getting ready
      2. How to do it...
      3. How it works...
    8. Enabling basic access authentication
      1. Getting ready
      2. How to do it...
      3. How it works...
    9. Using a custom authentication plugin
      1. Getting ready
      2. How to do it...
      3. How it works...
    10. Tuning Marathon
      1. Getting ready
      2. How to do it...
      3. How it works...
  8. Job Scheduling with Metronome
    1. Introduction
    2. Installing and configuring Metronome
      1. How to do it...
      2. How it works...
    3. Monitoring Metronome
      1. How to do it...
      2. How it works...
    4. Scheduling jobs
      1. How to do it...
      2. How it works...
  9. Continuous Integration with Jenkins
    1. Introduction
    2. Building the Jenkins Mesos plugin
      1. Getting ready
      2. How to do it...
      3. How it works...
    3. Installing Jenkins
      1. How to do it...
    4. Enabling the Jenkins Mesos plugin
      1. Getting ready
      2. How to do it...