Docker on Amazon Web Services

Docker on Amazon Web Services

by Justin Menga
Released August 2018
Publisher(s): Packt Publishing
ISBN: 9781788626507

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Run Docker on AWS and build real-world, secure, and scalable container platforms on cloud

Key Features

  • Configure Docker for the ECS environment
  • Integrate Docker with different AWS tools
  • Implement container networking and deployment at scale

Book Description

Over the last few years, Docker has been the gold standard for building and distributing container applications. Amazon Web Services (AWS) is a leader in public cloud computing, and was the first to offer a managed container platform in the form of the Elastic Container Service (ECS).

Docker on Amazon Web Services starts with the basics of containers, Docker, and AWS, before teaching you how to install Docker on your local machine and establish access to your AWS account. You'll then dig deeper into the ECS, a native container management platform provided by AWS that simplifies management and operation of your Docker clusters and applications for no additional cost. Once you have got to grips with the basics, you'll solve key operational challenges, including secrets management and auto-scaling your infrastructure and applications. You'll explore alternative strategies for deploying and running your Docker applications on AWS, including Fargate and ECS Service Discovery, Elastic Beanstalk, Docker Swarm and Elastic Kubernetes Service (EKS). In addition to this, there will be a strong focus on adopting an Infrastructure as Code (IaC) approach using AWS CloudFormation.

By the end of this book, you'll not only understand how to run Docker on AWS, but also be able to build real-world, secure, and scalable container platforms in the cloud.

What you will learn

  • Build, deploy, and operate Docker applications using AWS
  • Solve key operational challenges, such as secrets management
  • Exploit the powerful capabilities and tight integration of other AWS services
  • Design and operate Docker applications running on ECS
  • Deploy Docker applications quickly, consistently, and reliably using IaC
  • Manage and operate Docker clusters and applications for no additional cost

Who this book is for

Docker on Amazon Web Services is for you if you want to build, deploy, and operate applications using the power of containers, Docker, and Amazon Web Services. Basic understanding of containers and Amazon Web Services or any other cloud provider will be helpful, although no previous experience of working with these is required.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Docker on Amazon Web Services
  3. Dedication
  4. Packt Upsell
    1. Why subscribe?
    2. PacktPub.com
  5. Contributors
    1. About the author
    2. About the reviewer
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
    4. Code in Action
      1. Conventions used
    5. Get in touch
      1. Reviews
  7. Container and Docker Fundamentals
    1. Technical requirements
    2. Introduction to containers and Docker
      1. Why containers are revolutionary
      2. Docker architecture
      3. Running Docker in AWS
    3. Setting up a local Docker environment
      1. Setting up a macOS environment
        1. Installing other tools
      2. Setting up a Windows 10 environment
        1. Installing the Windows subsystem for Linux
        2. Installing Docker in the Windows subsystem for Linux
        3. Installing other tools in the Windows subsystem for Linux
      3. Setting up a Linux environment
    4. Installing the sample application
      1. Forking the sample application
      2. Running the sample application locally
        1. Installing application dependencies
        2. Running database migrations
        3. Running the local development web server
      3. Testing the sample application locally
    5. Summary
    6. Questions
    7. Further reading
  8. Building Applications Using Docker
    1. Technical requirements
    2. Testing and building the application using Docker
      1. Creating a test stage
        1. Installing system and build dependencies
        2. Installing application dependencies
        3. Copying application source and running tests
      2. Configuring the release stage
        1. Installing system dependencies
        2. Creating an application user
        3. Copying and installing application source code and dependencies
        4. Building and running the release image
    3. Testing and building the application using Docker Compose
      1. Adding a database service using Docker Compose
      2. Running database migrations
      3. Generating static web content
      4. Creating acceptance tests
    4. Automating the workflow
      1. Automating the test stage
      2. Automating the release stage
      3. Refining the workflow
        1. Cleaning up the Docker environment
        2. Using dynamic port mapping
        3. Adding a version target
      4. Testing the end-to-end workflow
    5. Summary
    6. Questions
    7. Further reading
  9. Getting Started with AWS
    1. Technical requirements
    2. Setting up an AWS account
      1. Installing Google Authenticator
    3. Logging in as the root account
    4. Creating IAM users, groups, and roles
      1. Creating IAM roles
      2. Creating an Administrators group
      3. Creating a Users group
      4. Creating an IAM user
      5. Logging in as an IAM user
      6. Enabling MFA for an IAM user
      7. Assuming an IAM role
    5. Creating an EC2 Key Pair
    6. Using the AWS CLI
      1. Installing the AWS CLI
      2. Creating an AWS access key
      3. Configuring the AWS CLI
      4. Configuring the AWS CLI to assume a role
      5. Configuring the AWS CLI to use a named profile
    7. Introduction to AWS CloudFormation
      1. Defining a CloudFormation template
      2. Deploying a CloudFormation stack
      3. Updating a CloudFormation Stack
      4. Deleting a CloudFormation stack
    8. Summary
    9. Questions
    10. Further reading
  10. Introduction to ECS
    1. Technical requirements
    2. ECS architecture
    3. Creating an ECS cluster
    4. Understanding ECS container instances
      1. Joining an ECS cluster
        1. Granting access to join an ECS cluster
      2. Managing ECS container instances
        1. Connecting to ECS container instances
        2. Inspecting the local Docker environment
        3. Inspecting the ECS agent
        4. Verifying the ECS agent
        5. ECS container instance logs
    5. Creating an ECS task definition
    6. Creating an ECS service
    7. Deploying ECS services
    8. Running ECS tasks
    9. Using the ECS CLI
    10. Deleting the Test Cluster
    11. Summary
    12. Questions
    13. Further information
  11. Publishing Docker Images Using ECR
    1. Technical requirements
    2. Understanding ECR
    3. Creating ECR repositories
      1. Creating ECR repositories using the AWS Console
      2. Creating ECR repositories using the AWS CLI
      3. Creating ECR repositories using AWS CloudFormation
    4. Logging into ECR
    5. Publishing Docker images to ECR
      1. Publishing Docker images using the Docker CLI
      2. Publishing Docker images using Docker Compose
      3. Automating the publish workflow
        1. Automating login and logout
        2. Automating the publishing of Docker images
    6. Pulling Docker images from ECR
      1. ECS container instance access to ECR from the same account
      2. ECS container instance access to ECR from a different account
        1. Configuring ECR resource policies using the AWS Console
        2. Configuring ECR resource policies using the AWS CLI
        3. Configuring ECR resource policies using AWS CloudFormation
        4. Configuring IAM policies in remote accounts
      3. AWS service access to ECR
    7. Configuring lifecycle policies
      1. Configuring lifecycle policies using the AWS Console
      2. Configuring lifecycle policies using the AWS CLI
      3. Configuring lifecycle policies using AWS CloudFormation
    8. Summary
    9. Questions
    10. Further reading
  12. Building Custom ECS Container Instances
    1. Technical requirements
    2. Designing a custom Amazon Machine Image
    3. Building a custom AMI using Packer
      1. Installing Packer
      2. Creating a Packer template
        1. Packer template structure
        2. Configuring a builder
        3. Configuring variables
        4. Configuring provisioners
        5. Configuring post-processors
      3. Building a machine image
        1. Generating dynamic session credentials
        2. Automating generation of dynamic session credentials
        3. Building the image
    4. Building custom ECS container instance images using Packer
      1. Defining a custom storage configuration
        1. Adding EBS volumes
        2. Formatting and mounting volumes
      2. Installing additional packages and configuring system settings
        1. Installing additional packages
        2. Configuring system settings
          1. Configuring timezone settings
          2. Modifying default cloud-init behavior
      3. Configuring a cleanup script
      4. Creating a first-run script
        1. Configuring ECS cluster membership
        2. Configuring HTTP proxy support
        3. Configuring the CloudWatch logs agent
        4. Starting required services
        5. Performing required health checks
    5. Testing your custom ECS container instance image
    6. Summary
    7. Questions
    8. Further reading
  13. Creating ECS Clusters
    1. Technical requirements
    2. Deployment overview
    3. Defining an ECS cluster
    4. Configuring an EC2 Auto Scaling group
      1. Creating an EC2 Auto Scaling group
      2. Configuring CloudFormation Input Parameters
    5. Defining an EC2 Auto Scaling launch configuration
    6. Configuring CloudFormation Init Metadata
    7. Configuring Auto Scaling group creation policies
    8. Configuring EC2 instance profiles
    9. Configuring EC2 security groups
    10. Deploying and testing an ECS cluster
    11. Summary
    12. Questions
    13. Further reading
  14. Deploying Applications Using ECS
    1. Technical requirements
    2. Creating an application database using RDS
      1. Configuring supporting RDS resources
      2. Deploying RDS resources using CloudFormation
    3. Configuring Application Load Balancers
      1. Application Load Balancer architecture
      2. Configuring an Application Load Balancer
        1. Creating an Application Load Balancer
        2. Configuring Application Load Balancer security groups
        3. Creating a listener
        4. Creating a target group
      3. Deploying an Application Load Balancer using CloudFormation
    4. Creating ECS task definitions
      1. Configuring ECS task definition families
      2. Configuring ECS task definition volumes
      3. Configuring ECS task definition containers
      4. Deploying ECS task definitions using CloudFormation
    5. Deploying ECS services
      1. Deploying an ECS service using CloudFormation
      2. ECS rolling deployments
      3. Executing a rolling deployment
    6. Creating a CloudFormation custom resource
      1. Understanding CloudFormation custom resources
      2. Creating a custom resource Lambda function
        1. Understanding the custom resource function code
        2. Understanding the custom resource Lambda function resources
      3. Creating custom resources
      4. Deploying custom resources
      5. Verifying the application
    7. Summary
    8. Questions
    9. Further reading
  15. Managing Secrets
    1. Technical requirements
    2. Creating KMS keys
      1. Encrypting and decrypting data using KMS
    3. Creating secrets using the AWS Secrets Manager
      1. Creating secrets using the AWS console
      2. Creating secrets using the AWS CLI
      3. Retrieving secrets using the AWS CLI
      4. Updating secrets using the AWS CLI
      5. Deleting and restoring secrets using the AWS CLI
    4. Injecting secrets at container startup
      1. Creating an entrypoint script
      2. Adding an entrypoint script to a Dockerfile
    5. Provisioning secrets using CloudFormation
      1. Configuring ECS task definitions to use secrets
      2. Exposing secrets to other resources
        1. Creating a Secrets Manager Lambda function
        2. Creating a secrets custom resource
    6. Deploying secrets to AWS
    7. Summary
    8. Questions
    9. Further reading
  16. Isolating Network Access
    1. Technical requirements
    2. Understanding ECS task networking
      1. Docker bridge networking
      2. ECS task networking
    3. Configuring a NAT gateway
      1. Configuring private subnets and route tables
      2. Configuring NAT gateways
      3. Configuring routing for your private subnets
    4. Configuring ECS task networking
      1. Configuring ECS task definitions for task networking
      2. Configuring ECS services for task networking
      3. Configuring supporting resources for task networking
    5. Deploying and testing ECS task networking
    6. Summary
    7. Questions
    8. Further reading
  17. Managing ECS Infrastructure Life Cycle
    1. Technical requirements
    2. Understanding ECS life cycle management
      1. EC2 Auto Scaling life cycle hooks
      2. ECS container instance draining
      3. ECS life cycle management solution
    3. Building a new ECS container instance AMI
    4. Configuring EC2 Auto Scaling rolling updates
    5. Creating EC2 Auto Scaling life cycle hooks
    6. Creating a Lambda function for consuming life cycle hooks
      1. Configuring permissions for the life cycle hook Lambda function
    7. Deploying and testing Auto Scaling life cycle hooks
    8. Summary
    9. Questions
    10. Further reading
  18. ECS Auto Scaling
    1. Technical requirements
    2. Understanding ECS cluster resources
      1. CPU resources
      2. Memory resources
      3. Network resources
    3. Calculating the ECS cluster capacity
      1. Calculating the container capacity
      2. Deciding when to scale out
      3. Calculating the idle host capacity
        1. Idle host capacity example
    4. Implementing an ECS Auto Scaling solution
      1. Configuring CloudWatch events for ECS
      2. Programming the Lambda function that calculates the cluster capacity
        1. Adding IAM permissions for calculating the cluster capacity
        2. Testing cluster-capacity calculations
        3. Publishing custom CloudWatch metrics
      3. Creating CloudWatch alarms for cluster-capacity management
      4. Creating EC2 Auto Scaling policies
      5. Testing ECS cluster-capacity management
        1. Testing scale out
        2. Testing scale in
      6. Configuring the AWS application Auto Scaling service
        1. Configuring CloudWatch alarms
        2. Defining an Auto Scaling target
        3. Creating an Auto Scaling IAM role
        4. Configuring scale-out and scale-in policies
      7. Deploying application Auto Scaling
    5. Summary
    6. Questions
    7. Further reading
  19. Continuously Delivering ECS Applications
    1. Technical requirements
    2. Introducing CodePipeline and CodeBuild
    3. Creating a custom CodeBuild container
      1. Defining a custom CodeBuild container
      2. Creating a repository for the custom CodeBuild container
    4. Adding CodeBuild support to your application repository
    5. Creating a continuous integration pipeline using CodePipeline
      1. Creating a CodePipeline pipeline using the AWS console
    6. Creating a continuous delivery pipeline using CodePipeline
      1. Publishing version information in your source repository
      2. Adding CodePipeline support to the deployment repository
      3. Creating an IAM role for CloudFormation deployments
      4. Adding a deployment repository to CodePipeline
      5. Adding an output artifact to the build stage
      6. Adding a deployment stage to the pipeline
    7. Continuously delivering to production using CodePipeline
      1. Adding a new environment configuration file to your deployment repository
      2. Adding a create change set action to the pipeline
      3. Adding a manual approval action to the pipeline
      4. Adding a deploy change set action to the pipeline
      5. Deploying to production
    8. Summary
    9. Questions
    10. Further reading
  20. Fargate and ECS Service Discovery
    1. Technical requirements
    2. When to use Fargate?
    3. Adding support for AWS X-Ray to applications
    4. Creating an X-Ray daemon Docker image
    5. Configuring ECS service discovery resources
      1. Configuring a service discovery namespace
      2. Configuring a service discovery service
    6. Configuring an ECS task definition for Fargate
      1. Configuring IAM roles for Fargate
    7. Configuring an ECS service for Fargate
    8. Deploying and testing the X-Ray daemon
      1. Configuring the todobackend stack for X-Ray support
      2. Testing the X-Ray service
    9. Summary
    10. Questions
    11. Further reading
  21. Elastic Beanstalk
    1. Technical requirements
    2. Introduction to Elastic Beanstalk
      1. Elastic Beanstalk concepts
    3. Creating an Elastic Beanstalk application
      1. Creating a Dockerrun.aws.json file
      2. Creating an Elastic Beanstalk application using the AWS console
      3. Configuring the EC2 instance profile
    4. Configuring Elastic Beanstalk applications using the CLI
      1. Managing Elastic Beanstalk EC2 instances
    5. Customizing Elastic Beanstalk applications
      1. Resolving Docker volume permissions issues
      2. Configuring database settings
      3. Running database migrations
    6. Summary
    7. Questions
    8. Further reading
  22. Docker Swarm in AWS
    1. Technical requirements
    2. Docker Swarm introduction
      1. Docker Swarm versus Kubernetes
    3. Installing Docker for AWS
      1. Resources created by the Docker for AWS CloudFormation stack
      2. Accessing the Swarm cluster
      3. Setting up local access to Docker Swarm
        1. Configuring SSH agent forwarding
        2. Configuring SSH tunneling
    4. Deploying applications to Docker Swarm
      1. Docker services
      2. Docker stacks
    5. Deploying the sample application to Docker Swarm
      1. Integrating Docker Swarm with the Elastic Container Registry
      2. Defining a stack
      3. Creating shared storage for hosting static content
      4. Creating a collectstatic service
      5. Creating persistent storage for storing the application database
        1. Relocating an EBS volume
      6. Secrets management using Docker secrets
        1. Configuring applications to consume secrets
      7. Running database migrations
    6. Summary
    7. Questions
    8. Further reading
  23. Elastic Kubernetes Service
    1. Technical requirements
    2. Introduction to Kubernetes
      1. Kubernetes versus Docker Swarm
      2. Kubernetes architecture
    3. Getting started with Kubernetes
      1. Creating a pod
      2. Creating a deployment
      3. Creating a service
      4. Exposing a service
      5. Adding volumes to your pods
      6. Adding init containers to your pods
      7. Adding a database service
        1. Creating persistent storage
        2. Creating a database service
      8. Creating and consuming secrets
        1. Consuming secrets for the database service
        2. Consuming secrets for the application
      9. Running jobs
    4. Creating an EKS cluster
      1. Installing client components
      2. Creating cluster resources
      3. Configuring kubectl for EKS
      4. Creating worker nodes
        1. Joining worker nodes to your EKS cluster
      5. Deploying the Kubernetes dashboard
    5. Deploying the sample application to EKS
      1. Configuring support for persistent volumes using AWS EBS
      2. Configuring support for AWS Elastic Load Balancers
      3. Deploying the sample application
        1. Creating secrets
        2. Deploying the database service
        3. Deploying the application service
        4. Tearing down down the sample application
    6. Summary
    7. Questions
    8. Further reading
  24. Assessments
    1. Chapter 1, Container and Docker Fundamentals
    2. Chapter 2, Building Applications Using Docker
    3. Chapter 3, Getting Started with AWS
    4. Chapter 4, Introduction to ECS
    5. Chapter 5, Publishing Docker Images Using ECR
    6. Chapter 6, Building Custom ECS Container Instances
    7. Chapter 7, Creating ECS Clusters
    8. Chapter 8, Deploying Applications Using ECS
    9. Chapter 9, Managing Secrets
    10. Chapter 10, Isolating Network Access
    11. Chapter 11, Managing the ECS Infrastructure Life Cycle
    12. Chapter 12, ECS Auto Scaling
    13. Chapter 13, Continuously Delivering ECS Applications
    14. Chapter 14, Fargate and ECS Service Discovery
    15. Chapter 15, Elastic Beanstalk
    16. Chapter 16, Docker Swarm in AWS
    17. Chapter 17, Elastic Kubernetes Service
  25. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Docker on Amazon Web Services
  • Author(s): Justin Menga
  • Release date: August 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781788626507