book

DevOps: Puppet, Docker, and Kubernetes

Name: DevOps: Puppet, Docker, and Kubernetes
ISBN: 9781788297615

by Thomas Uphill, John Arundel, Neependra Khare, Hideto Saito, Hui-Chuan Chloe Lee, Ke-Jou Carol Hsu

March 2017

Beginner to intermediate

925 pages

18h 11m

English

Packt Publishing

Read now

Unlock full access

DevOps: Puppet, Docker, and Kubernetes
Table of Contents
DevOps: Puppet, Docker, and Kubernetes
Credits
Preface
What this learning path covers
What you need for this learning path
Who this learning path is for
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions
1. Module 1

1. Puppet Language and Style
Introduction
Adding a resource to a node
How to do it...How it works...
Using Facter to describe a node
How to do it...How it works...There's more...VariablesScope
Installing a package before starting a service
How to do it...How it works...CapitalizationLearning metaparameters and orderingTrifectaIdempotency
Installing, configuring, and starting a service
How to do it...How it works…
Using community Puppet style
How to do it…IndentationQuotingFalseVariablesParametersSymlinks
Creating a manifest
How to do it...There's more…
Checking your manifests with Puppet-lint
Getting readyHow to do it...There's more...See also
Using modules
How to do it…How it works…There's more…TemplatesFacts, functions, types, and providersThird-party modulesModule organizationSee also
Using standard naming conventions
How to do it…There's more…
Using inline templates
How to do it…How it works…There's more...See also
Iterating over multiple items
How to do it…How it works…There's more…Using hashesCreating arrays with the split function
Writing powerful conditional statements
How to do it…How it works…There's more…Elseif branchesComparisonsCombining expressionsSee also
Using regular expressions in if statements
How to do it…How it works…There's more…Capturing patternsRegular expression syntaxSee also
Using selectors and case statements
How to do it…How it works…SelectorCase statementThere's more…Regular expressionsDefaults
Using the in operator
How to do it…There's more…
Using regular expression substitutions
How to do it…How it works…There's more…See also
Using the future parser
Getting readyHow to do it...Appending to and concatenating arraysLambda functionsReduceFilterMapSliceEachOther features
2. Puppet Infrastructure
Introduction
Installing Puppet
Getting readyHow to do it...
Managing your manifests with Git
Getting readyHow to do it...How it works...There's more...
Creating a decentralized Puppet architecture
Getting readyHow to do it...How it works...
Writing a papply script
How to do it...How it works...
Running Puppet from cron
Getting readyHow to do it...How it works...There's more...
Bootstrapping Puppet with bash
Getting readyHow to do it...How it works...There's more...
Creating a centralized Puppet infrastructure
Getting readyHow to do it...How it works...There's more...
Creating certificates with multiple DNS names
Getting readyHow to do it...How it works...
Running Puppet from passenger
Getting readyHow to do it...How it works...There's more...
Setting up the environment
Getting readyHow to do it...There's more...
Configuring PuppetDB
Getting readyHow to do it...How it works...There's more...
Configuring Hiera
Getting readyHow to do it...How it works...There's more...
Setting node-specific data with Hiera
Getting readyHow to do it...How it works...
Storing secret data with hiera-gpg
Getting readyHow to do it...How it works...There's more...See also
Using MessagePack serialization
Getting readyHow to do it...How it works...
Automatic syntax checking with Git hooks
How to do it...How it works...
Pushing code around with Git
Getting readyHow to do it...How it works...
Managing Environments with Git
Getting readyHow to do it...How it works...There's more...
3. Writing Better Manifests
Introduction
Using arrays of resources
How to do it…How it works…See also
Using resource defaults
How to do it...How it works...There's more...
Using defined types
How to do it…How it works…There's more…See also
Using tags
How to do it...There's more…
Using run stages
How to do it…How it works…There's more…See also
Using roles and profiles
How to do it…How it works…There's more…
Passing parameters to classes
How to do it…How it works…There's more…Specifying default values
Passing parameters from Hiera
Getting readyHow to do it...How it works...There's more...
Writing reusable, cross-platform manifests
How to do it…How it works...There's more…See also
Getting information about the environment
How to do it…How it works…There's more…See also
Importing dynamic information
Getting readyHow to do it…How it works…There's more…See also
Passing arguments to shell commands
How to do it…How it works…
4. Working with Files and Packages
Introduction
Making quick edits to config files
Getting readyHow to do it...How it works…There's more…
Editing INI style files with puppetlabs-inifile
Getting readyHow to do it...How it works...There's more...
Using Augeas to reliably edit config files
How to do it…How it works…There's more…
Building config files using snippets
Getting readyHow to do it...How it works...
Using ERB templates
How to do it…How it works…There's more…See also
Using array iteration in templates
How to do it…How it works…There's more…See also
Using EPP templates
How to do it...How it works...There's more...
Using GnuPG to encrypt secrets
Getting readyHow to do it...How it works...There's more...See also
Installing packages from a third-party repository
How to do it…How it works…There's more...
Comparing package versions
How to do it…How it works…
5. Users and Virtual Resources
Introduction
Using virtual resources
How to do it...How it works...There's more...
Managing users with virtual resources
How to do it...How it works...There's more...See also
Managing users' SSH access
How to do it...How it works...There's more...
Managing users' customization files
How to do it...How it works...There's more...See also
Using exported resources
Getting readyHow to do it...How it works...There's more...
6. Managing Resources and Files
Introduction
Distributing cron jobs efficiently
How to do it...How it works...There's more...See also
Scheduling when resources are applied
How to do it...How it works...There's more...
Using host resources
How to do it...How it works...There's more...
Using exported host resources
Getting readyHow to do it...How it works...There's more...
Using multiple file sources
How to do it...How it works...There's more...See also
Distributing and merging directory trees
How to do it...How it works...There's more...
Cleaning up old files
How to do it...How it works...There's more...
Auditing resources
How to do it...How it works...There's more...See also
Temporarily disabling resources
How to do it...How it works...
7. Managing Applications
Introduction
Using public modules
How to do it...How it works...There's more...
Managing Apache servers
How to do it...How it works...
Creating Apache virtual hosts
How to do it...How it works...There's more...
Creating nginx virtual hosts
How to do it...How it works...There's more...
Managing MySQL
How to do it...How it works...There's more...
Creating databases and users
How to do it...How it works...There's more...
8. Internode Coordination
Introduction
Managing firewalls with iptables
Getting readyHow to do it...How it works...There's more...
Building high-availability services using Heartbeat
Getting readyHow to do it…How it works…There's more...
Managing NFS servers and file shares
How to do it...How it works…
Using HAProxy to load-balance multiple web servers
How to do it…How it works…There's more...
Managing Docker with Puppet
Getting readyHow to do it...How it works...There's more...
9. External Tools and the Puppet Ecosystem
Introduction
Creating custom facts
How to do it...How it works...There's more...See also
Adding external facts
Getting readyHow to do it...How it works...There's more...Debugging external factsUsing external facts in PuppetSee also
Setting facts as environment variables
How to do it...
Generating manifests with the Puppet resource command
How to do it...There's more...
Generating manifests with other tools
Getting readyHow to do it...There's more...
Using an external node classifier
Getting readyHow to do it...How it works...There's more...See also
Creating your own resource types
How to do it...How it works...There's more...DocumentationValidation
Creating your own providers
How to do it...How it works...There's more...
Creating custom functions
How to do it...How it works...There's more...
Testing your puppet manifests with rspec-puppet
Getting readyHow to do it...How it works...There's more...See also
Using librarian-puppet
Getting readyHow to do it...How it works...There's more...
Using r10k
Getting readyHow to do it...How it works...There's more...
10. Monitoring, Reporting, and Troubleshooting
Introduction
Noop – the don't change anything option
How to do it...How it works...There's more...See also
Logging command output
How to do it...How it works...There's more...
Logging debug messages
How to do it...How it works...There's more...Printing out variable valuesResource ordering
Generating reports
How to do it...How it works...There's more...Other report typesSee also
Producing automatic HTML documentation
How to do it...How it works...There's more...
Drawing dependency graphs
Getting readyHow to do it...How it works...There's more...See also
Understanding Puppet errors
How to do it...See also
Inspecting configuration settings
How to do it...How it works...See also
2. Module 2
1. Introduction and Installation
IntroductionNamespacesThe pid namespaceThe net namespaceThe ipc namespaceThe mnt namespaceThe uts namespaceThe user namespaceCgroupsThe Union filesystem
Verifying the requirements for Docker installation
Getting readyHow to do it…How it works…See also
Installing Docker
Getting readyHow to do it…How it works...There's more…See also
Pulling an image and running a container
Getting readyHow to do it…How it works…There's more…See also
Adding a nonroot user to administer Docker
Getting readyHow to do it…How it works…
Setting up the Docker host with Docker Machine
Getting readyHow to do it…How it works…There's more…See also
Finding help with the Docker command line
Getting readyHow to do it…How it works…See also
2. Working with Docker Containers
Introduction
Listing/searching for an image
Getting readyHow to do it…How it works…There's more…See also
Pulling an image
Getting readyHow to do it…How it works…There's more…See also
Listing images
Getting readyHow to do it…How it works…There's more…See also
Starting a container
Getting readyHow to do it…How it works…There's more…See also
Listing containers
Getting readyHow to do it…How it works…There's more…See also
Looking at the logs of containers
Getting readyHow to do it…How it works…There's more…See also
Stopping a container
Getting readyHow to do it…How it works…There's more…See also
Deleting a container
Getting readyHow to do it…There's more…How it works…See also
Setting the restart policy on a container
Getting readyHow to do it…There's more…See also
Getting privileged access inside a container
Getting readyHow to do it…How it works…There's more…See also
Exposing a port while starting a container
Getting readyHow to do it…There's more…See also
Accessing the host device inside the container
Getting readyHow to do it…How it works…See also
Injecting a new process to a running container
Getting readyHow to do it…How it works…See also
Returning low-level information about a container
Getting readyHow to do it…How it works…There's more…See also
Labeling and filtering containers
Getting readyHow to do it…How it works…There's more…See also
3. Working with Docker Images
Introduction
Creating an account with Docker Hub
Getting readyHow to do it…How it works…See also
Creating an image from the container
Getting readyHow to do it…How it works…There's more…See also
Publishing an image to the registry
Getting readyHow to do it…How it works…There's more…See also
Looking at the history of an image
Getting readyHow to do it…How it works…There's more…See also
Deleting an image
Getting readyHow to do it…There's more…See also
Exporting an image
Getting readyHow to do it…There's more…See also
Importing an image
Getting readyHow to do it…See also
Building images using Dockerfiles
Getting readyHow to do it…How it works…There's more…See also
Building an Apache image – a Dockerfile example
Getting readyHow to do it…How it works…There's more…See also
Accessing Firefox from a container – a Dockerfile example
Getting readyHow to do it…How it works…There's more...See also
Building a WordPress image – a Dockerfile example
Getting readyHow to do it…How it works…There's more…See also
Setting up a private index/registry
Getting readyHow to do it…How it works…There's more…See also
Automated builds – with GitHub and Bitbucket
Getting readyHow to do it…How it works…There's more…See also
Creating the base image – using supermin
Getting readyHow to do it…How it works…There's more…See also
Creating the base image – using Debootstrap
Getting readyHow to do it…See also
Visualizing dependencies between layers
Getting readyHow to do it…How it works…There's more…See also
4. Network and Data Management for Containers
Introduction
Accessing containers from outside
Getting readyHow to do it…How it works…There's more…See also
Managing data in containers
Getting readyHow to do it...How it works…There's more…See also
Linking two or more containers
Getting readyHow to do it…How it works…There's more…See also
Developing a LAMP application by linking containers
Getting readyHow to do it…How it works…
Networking of multihost containers with Flannel
Getting readyHow to do it…How it works…See also
Assigning IPv6 addresses to containers
Getting readyHow to do it…How it works…There's more…See also
5. Docker Use Cases
Introduction
Testing with Docker
Getting readyHow to do it…How it works…There's more…
Doing CI/CD with Shippable and Red Hat OpenShift
Getting readyHow to do it…How it works…See also
Doing CI/CD with Drone
Getting readyHow to do it…How it works…There's more…See also
Setting up PaaS with OpenShift Origin
Getting readyHow to do it…How it works…There's more…See also
Building and deploying an app on OpenShift v3 from the source code
Getting readyHow to do it…How it works…There's more…See also
Configuring Docker as a hypervisor driver for OpenStack
Getting readyHow to do it…How it works...There's more…See also
6. Docker APIs and Language Bindings
Introduction
Configuring the Docker daemon remote API
Getting readyHow to do it…How it works…There's more…See also
Performing image operations using remote APIs
Getting readyHow to do it…How it works…There's more…See also
Performing container operations using remote APIs
Getting readyHow to do it…How it works…See also
Exploring Docker remote API client libraries
Getting readyHow to do it…How it works…There's more…
Securing the Docker daemon remote API
Getting readyHow to do it...How it works…There's more…
7. Docker Performance
Introduction
Benchmarking CPU performance
Getting readyHow to do it…How it works…There's more…See also
Benchmarking disk performance
Getting readyHow to do it…How it works…There's more…See also
Benchmarking network performance
Getting readyHow to do it…How it works…There's more…See also
Getting container resource usage using the stats feature
Getting readyHow to do it…How it works…See also
Setting up performance monitoring
Getting readyHow to do it…How it works…There's more…See also
8. Docker Orchestration and Hosting Platforms
Introduction
Running applications with Docker Compose
Getting readyHow to do it…How it works…There's more…See also
Setting up cluster with Docker Swarm
Getting readyHow to do it…How it works…There's more…See also
Setting up CoreOS for Docker orchestration
Getting readyHow to do it…How it works…There's more…See also
Setting up a Project Atomic host
Getting readyHow to do it…How it works…There's more…See also
Doing atomic update/rollback with Project Atomic
Getting readyHow to do it…How it works…See also
Adding more storage for Docker in Project Atomic
Getting readyHow to do it…How it works…There's more…See also
Setting up Cockpit for Project Atomic
Getting readyHow to do it…How it works…There's more…See also
Setting up a Kubernetes cluster
Getting readyHow to do it…How it works…There's more…See also
Scaling up and down in a Kubernetes cluster
Getting readyHow to do it…How it works…There's more…See also
Setting up WordPress with a Kubernetes cluster
Getting readyHow to do it…How it works…There's more…See also
9. Docker Security
Introduction
Setting Mandatory Access Control (MAC) with SELinux
Getting readyHow to do it…How it works…There's more…See also
Allowing writes to volume mounted from the host with SELinux ON
Getting readyHow to do it…How it works…See also
Removing capabilities to breakdown the power of a root user inside a container
Getting readyHow to do it...How it works…There's more...See also
Sharing namespaces between the host and the container
Getting readyHow to do it…How it works…There's more...See also
10. Getting Help and Tips and Tricks
Introduction
Starting Docker in debug mode
Getting readyHow to do it...How it works…
Building a Docker binary from the source
Getting readyHow to do it…How it works…There's more…See also
Building images without using cached layers
Getting readyHow to do it…How it works…There's more…
Building your own bridge for container communication
Getting readyHow to do it…How it works…There's more…See also
Changing the default execution driver of Docker
Getting readyHow to do it…How it works…See also
Selecting the logging driver for containers
Getting readyHow to do it…How it works…There's more…See also
Getting real-time Docker events for containers
Getting readyHow to do it…How it works…There's more…See also
3. Module 3
1. Building Your Own Kubernetes
Introduction
Exploring architecture
Getting readyHow to do it…Kubernetes masterAPI server (kube-apiserver)Scheduler (kube-scheduler)Controller manager (kube-controller-manager)Command Line Interface (kubectl)Kubernetes nodekubeletProxy (kube-proxy)How it works…etcdOverlay networkFlannelSee also
Preparing your environment
Getting readyHardware resourceOperating systemHow to do it…Kubernetes masterKubernetes nodesetcdSee also
Building datastore
How to do it…Red Hat Enterprise Linux 7 or CentOS 7Ubuntu Linux 15.10 Wily WerewolfOther LinuxDownload a binaryCreating a userInstall etcdHow it works…Auto startup scriptStartup script (systemd)Startup script (init)ConfigurationSee also
Creating an overlay network
Getting readyInstallationCentOS 7 or Red Hat Enterprise Linux 7Other Linux optionsHow to do it…Flannel networking configurationIntegrating with DockerHow it works…See also
Configuring master
Getting readyHow to do it…InstallationCentOS 7 or Red Hat Enterprise Linux 7Adding daemon dependencyOther Linux optionsVerificationSee also
Configuring nodes
Getting readyHow to do it…InstallationCentOS 7 or Red Hat Enterprise Linux 7Other Linux optionsVerificationSee also
Run your first container in Kubernetes
Getting readyHow to do it…Running an HTTP server (nginx)Exposing the port for external accessStopping the applicationHow it works…See also
2. Walking through Kubernetes Concepts
Introduction
An overview of Kubernetes control
Getting readyHow to do it…How it works…See also
Working with pods
Getting readyHow to do it…How it works…See also
Working with a replication controller
Getting readyHow to do it…Creating a replication controllerGetting information of a replication controllerChanging the configuration of a replication controllerRemoving a replication controllerHow it works…See also
Working with services
Getting readyHow to do it…Creating services for different resourcesCreating a service for a podCreating a service for the replication controller and adding an external IPCreating a no-selector service for an endpointCreating a service with session affinity based on another serviceCreating a service in a different typeCreating a service in NodePort typeDeleting a serviceHow it works…See also
Working with volumes
Getting readyHow to do it…emptyDirhostPathnfsglusterfsiscsiflockerrbdgitRepoawsElasticBlockStoregcePersistentDiskdownwardAPIThere's more…PersistentVolumeSee also
Working with secrets
Getting readyHow to do it…Creating a secretPicking up secret in the containerDeleting a secretHow it works…There's more…See also
Working with names
Getting readyHow to do it…How it works…See also
Working with namespaces
Getting readyHow to do it…Changing the default namespaceDeleting a namespaceThere's more…Deleting LimitRangeSee also
Working with labels and selectors
Getting readyHow to do it…Linking service with a replication controller by using label selectorsThere's more…See also
3. Playing with Containers
Introduction
Scaling your containers
Getting readyHow to do it…How it works…See also
Updating live containers
Getting readyHow to do it…How it works…There's more…See also
Forwarding container ports
Getting readyHow to do it…Container-to-container communicationsPod-to-pod communicationsPod-to-service communicationsExternal-to-internal communicationsSee also
Ensuring flexible usage of your containers
Getting readyHow to do it…Pod as a jobCreating a job with multiple pods runningPod as a daemon setRunning the daemon set only on specific nodesHow it works…See also
Working with configuration files
Getting readyYAMLJSONHow to do it…How it works…PodsReplication controllersServicesSee also
4. Building a High Availability Cluster
Introduction
Clustering etcd
Getting readyHow to do it…Staticetcd discoverySee also
Building multiple masters
Getting readyHow to do it…Preparing multiple master nodesSetting up kubelet in masterGetting the configuration files readyStarting the kubelet service and turning daemons on!How it works…See also
5. Building a Continuous Delivery Pipeline
Introduction
Moving monolithic to microservices
Getting readyHow to do it…MicroservicesFrontend WebUIHow it works…MicroservicesFrontend WebUISee also
Integrating with Jenkins
Getting readyInstall a Jenkins server which can build a Docker programHow to do it…Create your Jenkins projectRun a program testingDeploying a programHow it works…There's more…See also
Working with the private Docker registry
Getting readyHow to do it…How it works…AlternativesDocker Trusted RegistryNexus Repository ManagerAmazon EC2 Container RegistrySee also
Setting up the Continuous Delivery pipeline
Getting readyHow to do it…How it works…There's more…See also
6. Building Kubernetes on AWS
Introduction
Building the Kubernetes infrastructure in AWS
Getting readyHow to do it…VPC and subnetsInternet Gateway and NATRoute TableSecurity groupHow it works…See also
Managing applications using AWS OpsWorks
Getting readyHow to do it…The OpsWorks stackThe OpsWorks layerAdjusting the IAM roleThe OpsWorks instanceHow it works…See also
Auto-deploying Kubernetes through Chef recipes
Getting readyCreating ELB and its security groupsCreating an OpsWorks stackCreating application layersHow to do it…Stack configuration for custom recipesRecipes for etcdRecipes for the Kubernetes masterRecipes for the Kubernetes nodeStarting the instancesSee also
Using AWS CloudFormation for fast provisioning
Getting readyHow to do it…Creating a network infrastructureCreating OpsWorks for application managementSee also
7. Advanced Cluster Administration
Introduction
Advanced settings in kubeconfig
Getting readyHow to do it…Setting a new credentialSetting a new clusterSetting and changing the current contextCleaning up kubeconfigSee also
Setting resource in nodes
Getting readyHow to do it…Managing node capacityManaging computing resources in a podSee also
Playing with WebUI
Getting readyHow to do it…How it works…See also
Working with a RESTful API
Getting readyHow to do it…How it works…See also
Authentication and authorization
Getting readyHow to do it…Enabling authentication for an API callBasic authentication of etcdBasic authentication of the Kubernetes masterMaking use of user authorizationSee also
8. Logging and Monitoring
Introduction
Collecting application logs
Getting readyElasticsearchHow to do it…LogstashStartup scriptDockerfileDocker buildKubernetes replication controller and serviceKibanaHow it works…See also
Working with Kubernetes logs
Getting readyHow to do it…See also
Working with etcd log
Getting readyHow to do it…See also
Monitoring master and node
Getting readyUpdating Kubernetes to the latest version: 1.2.1Setting up the DNS serverStart the server using templatesEnable Kubernetes DNS in kubeletHow to do it…Installing a monitoring clusterIntroducing the Grafana dashboardCreating a new metric to monitor podsSee also
B. Bibliography
Index

Content preview from DevOps: Puppet, Docker, and Kubernetes

Working with Kubernetes logs

Kubernetes comes with three daemon processes on master: API server, scheduler, and controller manager. Under the /var/log folder, there are three corresponding log files recording the logs of these processes:

Daemon on master	Log file	Description
API server	`apiserver.log`	Logs for API calls.
Scheduler	`k8s-scheduler.log`	Logs of scheduler data for any containers scheduling events
Controller manager	`controller-manager.log`	Logs for showing any events or issues relate to controller manager

On nodes, we have a kubelet process to handle container operations and report to the master:

Daemon on node	Log file	Description
`kubelet`	`kubelet.log`	Logs for any issues happening in container

On both masters ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781788297615Purchase Link

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

DevOps: Puppet, Docker, and Kubernetes

by Thomas Uphill, John Arundel, Neependra Khare, Hideto Saito, Hui-Chuan Chloe Lee, Ke-Jou Carol Hsu

Working with Kubernetes logs

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

More than 5,000 organizations count on O’Reilly

Julian F.

Addison B.

Amir M.

Mark W.

You might also like

Cloud Native DevOps with Kubernetes

Hands-On Microservices with Kubernetes

Hands-On Kubernetes on Windows

Bootstrapping Microservices with Docker, Kubernetes, and Terraform

Publisher Resources