O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Native Docker Clustering with Swarm

Book Description

Deploy, configure, and run clusters of Docker containers with Swarm

About This Book

  • Get to grips with Docker Swarm, one of the key components of the Docker ecosystem.
  • Optimize Swarm and SwarmKit features for scaling massive applications through containers.
  • Learn about Docker’s scheduling tricks, high availability, security, and platform scalability.

Who This Book Is For

If you are a Linux admin or a Docker user who wants to natively manage Docker clusters, then this is the book for you.

What You Will Learn

  • Create and manage Swarm Mode clusters of any size
  • Get a backstage view of the biggest Swarms ever built : Swarm2k and Swarm3k, with their 2,300 and 4,700 nodes
  • Discovery mechanisms and Raft
  • Deploy your containerized app on Swarm
  • Administer Swarm clusters on AWS, Azure, and DigitalOcean
  • Integrate Flocker volumes with Swarm
  • Create and manage Swarms on OpenStack Magnum

In Detail

Docker Swarm serves as one of the crucial components of the Docker ecosystem and offers a native solution for you to orchestrate containers. It’s turning out to be one of the preferred choices for Docker clustering thanks to its recent improvements.

This book covers Swarm, Swarm Mode, and SwarmKit. It gives you a guided tour on how Swarm works and how to work with Swarm. It describes how to set up local test installations and then moves to huge distributed infrastructures. You will be shown how Swarm works internally, what’s new in Swarmkit, how to automate big Swarm deployments, and how to configure and operate a Swarm cluster on the public and private cloud.

This book will teach you how to meet the challenge of deploying massive production-ready applications and a huge number of containers on Swarm. You'll also cover advanced topics that include volumes, scheduling, a Libnetwork deep dive, security, and platform scalability.

Style and approach

A comprehensive guide that covers all aspects of Docker Swarm from setup to customization.

Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the code file.

Table of Contents

  1. Native Docker Clustering with Swarm
    1. Native Docker Clustering with Swarm
    2. Credits
    3. About the Authors
    4. About the Reviewer
    5. www.PacktPub.com
      1. Why subscribe?
    6. Dedication
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Errata
        2. Piracy
        3. Questions
    8. 1. Welcome to Docker Swarm
      1. Clustering tools and container managers
      2. Swarm goals
      3. Why use Swarm
      4. Real world use case examples
        1. Pet versus cattle models
      5. Swarm features
      6. Similar projects
        1. Kubernetes
        2. CoreOS Fleet
        3. Apache Mesos
        4. Kubernetes versus Fleet versus Mesos
        5. Swarm versus all
      7. The Swarm v1 architecture
        1. Terminology
      8. Getting started with Swarm
        1. Docker for Mac
        2. Docker for Windows
        3. Getting ready with Linux
        4. Check that Docker Machine is available - all systems
      9. Swarm, yesterday
        1. Boot2Docker
        2. Create 4 cluster nodes with Docker Machine
        3. Configuring the Docker hosts
        4. Starting Docker Swarm
      10. Test your Swarm cluster
      11. Swarm, today
      12. Summary
    9. 2. Discover the Discovery Services
      1. A discovery service
      2. Token
        1. Re-architecting the example of Chapter 1 with token
        2. Token limitations
      3. Raft
        1. Raft theory
        2. Raft in practice
      4. Etcd
        1. Re architecting the example of Chapter 1 with Etcd
      5. ZooKeeper
      6. Consul
        1. Re architecting the example of Chapter 1 with Consul
      7. Towards a decentralized discovery service
      8. Summary
    10. 3. Meeting Docker Swarm Mode
      1. Swarmkit
        1. Versions and support
        2. Swarmkit architecture
          1. How a manager chooses the best node for a task
        3. The heart of SwarmKit: swarmd
        4. The controller of SwarmKit: swarmctl
        5. Provisioning a SwarmKit cluster with Ansible
        6. Creating a service on SwarmKit
      2. Swarm mode
        1. Old versus new Swarm versus SwarmKit
        2. Swarm Mode zoom in
          1. Integration into the engine
            1. docker swarm command
            2. docker node
            3. docker service
            4. Docker Stack
          2. Etcd's Raft is integrated already
          3. Load balancing and DNS
          4. Promotion and demotion
          5. Replicas and scale
          6. Services and tasks
            1. docker service scale
      3. Summary
    11. 4. Creating a Production-Grade Swarm
      1. Tools
      2. An HA Topology for Swarm2k
        1. Managers specifications
          1. In case of Raft recovery
        2. Raft files
        3. Running tasks
        4. Manager topologies
      3. Provisioning the infrastructure with belt
      4. Securing Managers with Docker Machine
      5. Understanding some Swarm internals
        1. Joining workers
        2. Upgrading Managers
          1. Live Upgrading the Managers
      6. Monitoring Swarm2k
        1. InfluxDB Time-Series Database
          1. Telegraf Swarm plugin
      7. Swarm3k
        1. Swarm3k Setup and Workload
        2. Swarm performance at a scale
      8. Swarm2k and Swarm3k lessons learned
      9. Summary
    12. 5. Administer a Swarm Cluster
      1. Docker Swarm standalone
      2. Docker Swarm Mode
        1. Manually adding nodes
        2. Managers
        3. Workers number
        4. Scripted nodes addition
        5. Belt
        6. Use Ansible
      3. Cluster management
        1. Nodes operations
        2. Demotion and promotion
        3. Tagging nodes
        4. Remove nodes
          1. Remove workers
          2. Remove managers
      4. Swarm health
      5. Backing up the cluster configuration
      6. Disaster recovery
      7. Graphical interfaces for Swarm
        1. Shipyard
        2. Portainer
      8. Summary
    13. 6. Deploy Real Applications on Swarm
      1. Microservices
      2. Deploy a replicated nginx
        1. A minimal Swarm
        2. Docker service
      3. Overlay networks
        1. Integrated load balancing
      4. Connecting services: A WordPress example
        1. Swarm scheduling strategies
        2. Now, WordPress
      5. Docker Compose and Swarm mode
      6. Introducing Docker stacks
        1. Distributed Application Bundles
        2. Docker deploy
      7. Another app: Apache Spark
        1. Why Spark on Docker
        2. Spark standalone without Swarm
        3. Spark standalone on Swarm
          1. Spark topology
          2. Storage
          3. Prerequisites
        4. Start Spark on Swarm
      8. Summary
    14. 7. Scaling Up Your Platform
      1. The Spark example, again
      2. Docker plugins
      3. The lab
        1. A unique key
        2. Docker Machine
        3. Security groups
        4. Networking configuration
        5. Storage configuration and architecture
      4. Installing Flocker
        1. Generating Flocker certificates
        2. Installing software
        3. Installing the control node
        4. Installing the cluster nodes
        5. Testing whether everything is up and running
      5. Installing and configuring Swarm
        1. A volume for Spark
      6. Deploying Spark, again
        1. Testing Spark
        2. Using Flocker storage
      7. Scaling Spark
      8. Monitoring Swarm hosting apps
        1. Prometheus
        2. Installing a monitoring system
        3. Importing Prometheus in Grafana
      9. Summary
    15. 8. Exploring Additional Features of Swarm
      1. Libnetwork
        1. Networking plugins
        2. Container Networking Model
      2. Encryption and routing mesh
      3. MacVLAN
        1. Overlay networks
      4. Network Control Plane
      5. Libkv
        1. How to use libkv
      6. Summary
    16. 9. Securing a Swarm Cluster and the Docker Software Supply Chain
      1. Software Supply Chain
      2. Securing Swarm cluster
      3. Securing a Swarm: Best practices
        1. Certification Authorities
        2. Certificates and Mutual TLS
        3. The join token
        4. Add TLS with Docker Machine
          1. Form a cluster on a private network
      4. Docker Notary
      5. Introducing Docker secrets
      6. Summary
    17. 10. Swarm and the Cloud
      1. Docker for AWS and Azure
        1. Docker for AWS
        2. Docker for Azure
      2. Docker Datacenter
      3. Swarm on OpenStack
        1. OpenStack Nova
          1. The (deprecated) nova-docker driver
        2. The reality - OpenStack the friendly way
        3. OpenStack Heat
        4. OpenStack Magnum
          1. Architecture and core concepts
          2. Install HA Magnum on Mirantis OpenStack
          3. Configure an HA Magnum installation
          4. Create a Swarm cluster on Magnum
      4. Summary
    18. 11. What is next?
      1. The challenge of provisioning
      2. Software defined infrastructure
        1. Infrakit
        2. TUF - The Update Framework
      3. Docker stacks and Compose
      4. CaaS - Containers as a Service
      5. Unikernels
      6. Contribute to Docker
        1. Github
        2. File issues
        3. Code
        4. Belt and other projects
      7. Summary