book

Pipeline as Code

by Mohamed Labouardy

October 2021

Intermediate to advanced

528 pages

12h 44m

English

Manning Publications

Read now

Unlock full access

prefaceacknowledgmentsabout this bookWho should read this bookHow this book is organizedAbout the codeliveBook discussion forumOther online resourcesabout the authorabout the cover illustration
1.1 Going cloud native1.1.1 Monolithic1.1.2 Microservices1.1.3 Cloud native1.1.4 Serverless1.2 Defining continuous integration1.3 Defining continuous deployment1.4 Defining continuous delivery1.5 Embracing CI/CD practices1.6 Using essential CI/CD tools1.6.1 Choosing a CI/CD tool1.6.2 Introducing JenkinsSummary
2.1 Introducing the Jenkinsfile2.1.1 Blue Ocean plugin2.1.2 Scripted pipeline2.1.3 Declarative pipeline2.2 Understanding multibranch pipelines2.3 Exploring the GitFlow branch model2.4 Test-driven development with Jenkins2.4.1 The Jenkins Replay button2.4.2 Command-line pipeline linter2.4.3 IDE integrationsSummary

3.1 Understanding master-worker architecture3.2 Managing Jenkins workers3.2.1 SSH3.2.2 Command line3.2.3 JNLP3.2.4 Windows service3.3 Architecting Jenkins for scale in AWS3.3.1 Preparing the AWS environment3.3.2 Configuring the AWS CLI3.3.3 Creating and managing the IAM userSummary
4.1 Immutable infrastructure4.2 Introducing Packer4.2.1 How does it work?4.2.2 Installation and configuration4.2.3 Baking a machine image4.3 Baking the Jenkins master AMI4.3.1 Configuring Jenkins upon startup4.3.2 Discovering Jenkins plugins4.4 Baking the Jenkins worker AMISummary
5.1 Introducing infrastructure as code5.1.1 Terraform usage5.2 Provisioning an AWS VPC5.2.1 AWS VPC5.2.2 VPC subnets5.2.3 VPC route tables5.2.4 VPC bastion host5.3 Setting up a self-healing Jenkins master5.4 Running Jenkins with native SSL/HTTPS5.5 Dynamically autoscaling the Jenkins worker pool5.5.1 Launch configuration5.5.2 Auto Scaling group5.5.3 Autoscaling scaling policies5.5.4 Workers CPU utilization loadSummary
6.1 Google Cloud Platform6.1.1 Building Jenkins VM images6.1.2 Configuring a GCP network with Terraform6.1.3 Deploying Jenkins on Google Compute Engine6.1.4 Launching automanaged workers on GCP6.2 Microsoft Azure6.2.1 Building golden Jenkins VM images in Azure6.2.2 Deploying a private virtual network6.2.3 Deploying a Jenkins master virtual machine6.2.4 Applying autoscaling to Jenkins workers6.3 DigitalOcean6.3.1 Creating Jenkins DigitalOcean Snapshots6.3.2 Deploying a Jenkins master Droplet6.3.3 Building Jenkins worker DropletsSummary
7.1 Introducing microservices-based applications7.2 Defining multibranch pipeline jobs7.3 Git and GitHub integration7.4 Discovering Jenkins jobs’ XML configuration7.5 Configuring SSH authentication with Jenkins7.6 Triggering Jenkins builds with GitHub webhooksSummary
8.1 Running unit tests inside Docker containers8.2 Automating code linter integration with Jenkins8.3 Generating code coverage reports8.4 Injecting security in the CI pipeline8.5 Running parallel tests with Jenkins8.6 Improving quality with code analysis8.7 Running mocked database tests8.8 Generating HTML coverage reports8.9 Automating UI testing with Headless Chrome8.10 Integrating SonarQube Scanner with JenkinsSummary
9.1 Building Docker images9.1.1 Using the Docker DSL9.1.2 Docker build arguments9.2 Deploying a Docker private registry9.2.1 Nexus Repository OSS9.2.2 Amazon Elastic Container Registry9.2.3 Azure Container Registry9.2.4 Google Container Registry9.3 Tagging Docker images the right way9.4 Scanning Docker images for vulnerabilities9.5 Writing a Jenkins declarative pipeline9.6 Managing pull requests with JenkinsSummary
10.1 Running a distributed Docker Swarm cluster10.2 Defining a continuous deployment process10.3 Integrating Jenkins with Slack notifications10.4 Handling code promotion with Jenkins10.5 Implementing the Jenkins delivery pipelineSummary
11.1 Setting up a Kubernetes cluster11.2 Automating continuous deployment flow with Jenkins11.2.1 Migrating Docker Compose to K8s manifests with Kompose11.3 Walking through continuous delivery steps11.4 Packaging Kubernetes applications with Helm11.5 Running post-deployment smoke tests11.6 Discovering Jenkins XSummary
12.1 Deploying a Lambda-based application12.2 Creating deployment packages12.2.1 Mono-repo strategy12.2.2 Multi-repo strategy12.3 Updating Lambda function code12.4 Hosting a static website on S312.5 Maintaining multiple Lambda environments12.6 Configuring email notification in JenkinsSummary
13.1 Monitoring Jenkins cluster health13.2 Centralized logging for Jenkins logs with ELK13.2.1 Streaming logs with Filebeat13.2.2 Streaming logs with the Logstash plugin13.3 Creating alerts based on metricsSummary
14.1 Exploring Jenkins security and RBAC authorization14.1.1 Matrix authorization strategy14.1.2 Role-based authorization strategy14.2 Configuring GitHub OAuth for Jenkins14.3 Keeping track of Jenkins users’ actions14.4 Extending Jenkins with shared libraries14.5 Backing up and restoring Jenkins14.6 Setting up cron jobs with Jenkins14.7 Running Jenkins locally as a Docker containerSummaryWrapping up

Content preview from Pipeline as Code

4 Baking machine images with Packer

This chapter covers

Overview of immutable infrastructure
Baking Jenkins machine images with Packer
Discovering Jenkins essentials plugins
Executing Jenkins Groovy scripts
Using Packer provisioners to automate Jenkins settings

In the previous chapter, you learned how Jenkins distributed mode architecture works. In this one, we will get our hands dirty and deploy a Jenkins cluster on AWS. As a quick reminder, you learned that the Jenkins cluster is divided into two main components: master and worker. Before diving into the implementation of the distributed builds architecture, we will deploy the standalone mode, shown in figure 4.1, to cover some basics.

To deploy this architecture, we need to provision a ...