book

Pipeline as Code

by Mohamed Labouardy

October 2021

Intermediate to advanced

528 pages

12h 44m

English

Manning Publications

Read now

Unlock full access

prefaceacknowledgmentsabout this bookWho should read this bookHow this book is organizedAbout the codeliveBook discussion forumOther online resourcesabout the authorabout the cover illustration
1.1 Going cloud native1.1.1 Monolithic1.1.2 Microservices1.1.3 Cloud native1.1.4 Serverless1.2 Defining continuous integration1.3 Defining continuous deployment1.4 Defining continuous delivery1.5 Embracing CI/CD practices1.6 Using essential CI/CD tools1.6.1 Choosing a CI/CD tool1.6.2 Introducing JenkinsSummary
2.1 Introducing the Jenkinsfile2.1.1 Blue Ocean plugin2.1.2 Scripted pipeline2.1.3 Declarative pipeline2.2 Understanding multibranch pipelines2.3 Exploring the GitFlow branch model2.4 Test-driven development with Jenkins2.4.1 The Jenkins Replay button2.4.2 Command-line pipeline linter2.4.3 IDE integrationsSummary

3.1 Understanding master-worker architecture3.2 Managing Jenkins workers3.2.1 SSH3.2.2 Command line3.2.3 JNLP3.2.4 Windows service3.3 Architecting Jenkins for scale in AWS3.3.1 Preparing the AWS environment3.3.2 Configuring the AWS CLI3.3.3 Creating and managing the IAM userSummary
4.1 Immutable infrastructure4.2 Introducing Packer4.2.1 How does it work?4.2.2 Installation and configuration4.2.3 Baking a machine image4.3 Baking the Jenkins master AMI4.3.1 Configuring Jenkins upon startup4.3.2 Discovering Jenkins plugins4.4 Baking the Jenkins worker AMISummary
5.1 Introducing infrastructure as code5.1.1 Terraform usage5.2 Provisioning an AWS VPC5.2.1 AWS VPC5.2.2 VPC subnets5.2.3 VPC route tables5.2.4 VPC bastion host5.3 Setting up a self-healing Jenkins master5.4 Running Jenkins with native SSL/HTTPS5.5 Dynamically autoscaling the Jenkins worker pool5.5.1 Launch configuration5.5.2 Auto Scaling group5.5.3 Autoscaling scaling policies5.5.4 Workers CPU utilization loadSummary
6.1 Google Cloud Platform6.1.1 Building Jenkins VM images6.1.2 Configuring a GCP network with Terraform6.1.3 Deploying Jenkins on Google Compute Engine6.1.4 Launching automanaged workers on GCP6.2 Microsoft Azure6.2.1 Building golden Jenkins VM images in Azure6.2.2 Deploying a private virtual network6.2.3 Deploying a Jenkins master virtual machine6.2.4 Applying autoscaling to Jenkins workers6.3 DigitalOcean6.3.1 Creating Jenkins DigitalOcean Snapshots6.3.2 Deploying a Jenkins master Droplet6.3.3 Building Jenkins worker DropletsSummary
7.1 Introducing microservices-based applications7.2 Defining multibranch pipeline jobs7.3 Git and GitHub integration7.4 Discovering Jenkins jobs’ XML configuration7.5 Configuring SSH authentication with Jenkins7.6 Triggering Jenkins builds with GitHub webhooksSummary
8.1 Running unit tests inside Docker containers8.2 Automating code linter integration with Jenkins8.3 Generating code coverage reports8.4 Injecting security in the CI pipeline8.5 Running parallel tests with Jenkins8.6 Improving quality with code analysis8.7 Running mocked database tests8.8 Generating HTML coverage reports8.9 Automating UI testing with Headless Chrome8.10 Integrating SonarQube Scanner with JenkinsSummary
9.1 Building Docker images9.1.1 Using the Docker DSL9.1.2 Docker build arguments9.2 Deploying a Docker private registry9.2.1 Nexus Repository OSS9.2.2 Amazon Elastic Container Registry9.2.3 Azure Container Registry9.2.4 Google Container Registry9.3 Tagging Docker images the right way9.4 Scanning Docker images for vulnerabilities9.5 Writing a Jenkins declarative pipeline9.6 Managing pull requests with JenkinsSummary
10.1 Running a distributed Docker Swarm cluster10.2 Defining a continuous deployment process10.3 Integrating Jenkins with Slack notifications10.4 Handling code promotion with Jenkins10.5 Implementing the Jenkins delivery pipelineSummary
11.1 Setting up a Kubernetes cluster11.2 Automating continuous deployment flow with Jenkins11.2.1 Migrating Docker Compose to K8s manifests with Kompose11.3 Walking through continuous delivery steps11.4 Packaging Kubernetes applications with Helm11.5 Running post-deployment smoke tests11.6 Discovering Jenkins XSummary
12.1 Deploying a Lambda-based application12.2 Creating deployment packages12.2.1 Mono-repo strategy12.2.2 Multi-repo strategy12.3 Updating Lambda function code12.4 Hosting a static website on S312.5 Maintaining multiple Lambda environments12.6 Configuring email notification in JenkinsSummary
13.1 Monitoring Jenkins cluster health13.2 Centralized logging for Jenkins logs with ELK13.2.1 Streaming logs with Filebeat13.2.2 Streaming logs with the Logstash plugin13.3 Creating alerts based on metricsSummary
14.1 Exploring Jenkins security and RBAC authorization14.1.1 Matrix authorization strategy14.1.2 Role-based authorization strategy14.2 Configuring GitHub OAuth for Jenkins14.3 Keeping track of Jenkins users’ actions14.4 Extending Jenkins with shared libraries14.5 Backing up and restoring Jenkins14.6 Setting up cron jobs with Jenkins14.7 Running Jenkins locally as a Docker containerSummaryWrapping up

Overview

Start thinking about your development pipeline as a mission-critical application. Discover techniques for implementing code-driven infrastructure and CI/CD workflows using Jenkins, Docker, Terraform, and cloud-native services.

In Pipeline as Code, you will master:

Building and deploying a Jenkins cluster from scratch
Writing pipeline as code for cloud-native applications
Automating the deployment of Dockerized and Serverless applications
Containerizing applications with Docker and Kubernetes
Deploying Jenkins on AWS, GCP and Azure
Managing, securing and monitoring a Jenkins cluster in production
Key principles for a successful DevOps culture

Pipeline as Code is a practical guide to automating your development pipeline in a cloud-native, service-driven world. You’ll use the latest infrastructure-as-code tools like Packer and Terraform to develop reliable CI/CD pipelines for numerous cloud-native applications. Follow this book's insightful best practices, and you’ll soon be delivering software that’s quicker to market, faster to deploy, and with less last-minute production bugs.

About the Technology
Treat your CI/CD pipeline like the real application it is. With the Pipeline as Code approach, you create a collection of scripts that replace the tedious web UI wrapped around most CI/CD systems. Code-driven pipelines are easy to use, modify, and maintain, and your entire CI pipeline becomes more efficient because you directly interact with core components like Jenkins, Terraform, and Docker.

About the Book
In Pipeline as Code you’ll learn to build reliable CI/CD pipelines for cloud-native applications. With Jenkins as the backbone, you’ll programmatically control all the pieces of your pipeline via modern APIs. Hands-on examples include building CI/CD workflows for distributed Kubernetes applications, and serverless functions. By the time you’re finished, you’ll be able to swap manual UI-based adjustments with a fully automated approach!

What's Inside

Build and deploy a Jenkins cluster on scale
Write pipeline as code for cloud-native applications
Automate the deployment of Dockerized and serverless applications
Deploy Jenkins on AWS, GCP, and Azure
Grasp key principles of a successful DevOps culture

About the Reader
For developers familiar with Jenkins and Docker. Examples in Go.

About the Author
Mohamed Labouardy is the CTO and co-founder of Crew.work, a Jenkins contributor, and a DevSecOps evangelist.

Quotes
Configuring CI/CD platforms has never been easier!
- Ubaldo Pescatore, PagoPA

A must-read for any aspiring and seasoned devops/release automation engineer.
- Giridharan Kesavan, Visa

A very useful resource, not only for setting up and using Jenkins for CI/CD, but also for understanding the importance of Packer, Terraform, Docker, and Kubernetes.
- Kosmas Chatzimichalis, Mach7x

A perfect journey through pipeline-based software delivery.
- Satej Kumar Sahu, Honeywell

A brilliant, hands-on deep dive into how to implement modern CI/CD pipelines.
- Matthias Busch, Otto GmbH