Building a deployment pipeline with Jenkins 2
Published by O'Reilly Media, Inc.
Manage continuous integration and continuous delivery to release software
Jenkins is one of the most popular tools for implementing continuous delivery pipelines, and many key processes are setup in Jenkins pipelines composed of a set of freestyle jobs. Jenkins 2 brings a number of improvements, including a new style of programming interface: Groovy scripts with a special DSL and pipeline structure. If you use Jenkins, you need to know how to use this new technology.
Over two three-hour sessions, Brent Laster walks you through implementing a continuous delivery pipeline using Jenkins 2. Along the way, you'll explore Jenkins 2 basic models and concepts and learn how to integrate Jenkins 2 with key technologies such as Git, Gradle, SonarQube, JaCoCo, Artifactory, and Docker.
What you’ll learn and how you can apply it
By the end of this live, online course, you’ll understand:
- The Jenkins and Jenkins 2 models
- How Jenkins 2 is intended to be used
- How to create a continuous delivery pipeline in Jenkins 2
- How to integrate Jenkins 2 with other applications and tools, such as Git, Gradle, and Artifactory
And you’ll be able to:
- Implement a continuous delivery pipeline on your own
- Get started with Jenkins 2 in your own projects
This live event is for you because...
- You're a developer who wants to have your code changes validated quickly and efficiently through automated pipeline processes.
- You're a manager or project manager who needs to quickly get your code tested and validated and to a deployable state.
- You're a tester who wants to automate testing and QA in your projects.
Prerequisites
- A basic understanding of source control concepts
- Familiarity with Jenkins (useful but not required)
Required materials and setup:
- A machine with VirtualBox installed
Prior to class, you will need to download the Virtual Machine image and follow the setup instructions provided below. The setup instructions contain the link/url of the labs for the course, which you will want to have available to copy and paste code.
Setup instructions: https://github.com/brentlaster/safaridocs/blob/master/bdpj2-setup.pdf
Virtual machine image:
https://www.dropbox.com/s/a76njwon38db6js/BDPJ2v3.ova?dl=0
or you can download the machine image from: https://bclconf.s3-us-west-2.amazonaws.com/BDPJ2v3.ova
- A GitHub account (free tier)
Recommended preparation:
"Dockerizing Jenkins 2: Part 1—Declarative Build Pipeline with SonarQube Analysis" (article)
Building a Deployment Pipeline with Jenkins 2: Part 1 (video)
Building a Deployment Pipeline with Jenkins 2: Part 2 (video)
Building a Deployment Pipeline with Jenkins 2: Part 3 (video)
Schedule
The time frames are only estimates and may vary according to how the class is progressing.
Day 1
Continuous delivery and deployment pipeline core concepts (20 minutes)
- Lecture: Key concepts for continuous delivery and deployment pipelines; components of a deployment pipeline and the theory and practices behind each
Introduction to Jenkins and pipelines (20 minutes)
- Lecture: Overview of the Jenkins object model (how the main parts of Jenkins are structured and how they fit together); Groovy pipeline programming in Jenkins; the differences between scripted and declarative pipelines; the pipeline editor
Jenkins nodes (10 minutes)
- Lecture: What a node is, how to set it up, and how to specify it in a pipeline script; how to use labels to identify nodes; nodes versus slave nodes versus agents; how nodes can be used and configured
Hands-on exercise (10 minutes)
- Set up a new node on your virtual image to use throughout the rest of the class
- Break (10 minutes)
Overview of Git and Gradle (10 minutes)
- Lecture: Git fundamental concepts and usage in a pipeline; Gradle fundamental concepts and usage in a pipeline
Global tools and the pipeline (5 minutes)
- Lecture: How to define global tools in Jenkins; how to get the global values of tools from Jenkins from within a pipeline and how to use the tools in the pipeline; shell calls and DSL calls to invoke functionality
DSL syntax and the Snippet Generator (5 minutes)
- Lecture: DSL syntax; the Snippet Generator and how it helps with generating correct syntax for a pipeline
Shared pipeline libraries (15 minutes)
- Lecture: Shared pipeline libraries—structure, uses, and how to create and maintain them; submodules—how to update them, their benefits, how to keep them in sync, and how they relate to other Git projects; how to include, load, and invoke libraries and functions within a pipeline script; stage view; the replay feature
Hands-on exercise (15 minutes)
- Add stages, configure tools, and use shared libraries to create a function
- Break (10 minutes)
Running testing operations in parallel (15 minutes)
- Lecture: The parallel DSL operation; how to map specific tasks to run in parallel in the pipeline; using the stash and unstash commands to share files and data needed across parallel operations running on separate nodes; how to use a statement to clean out workspaces before executing tasks
Using credentials with a pipeline script (5 minutes)
- Lecture: How to define global credentials in Jenkins; how to pass username and password credentials to an operation in the pipeline; how to pass SSH credentials to an operation in the pipeline
Leveraging source sets (15 minutes)
- Lecture: Using Gradle source sets to provide an additional test context like unit testing but for integration tests
Hands-on exercise (15 minutes)
- Utilize the parallel mapping structure to add unit tests that run in parallel to the pipeline; add integration testing into the pipeline using credentials and sourcesets
Day 2
Working with SonarQube (15 minutes)
- Lecture: SonarQube—what it is, what it does, and how to integrate it into the Jenkins pipeline using a combination of DSL functions provided by the SonarQube plugin and the SonarQube webhook functionality
Pipeline flow control (5 minutes)
- Lecture: Available parallel flow control DSL operations options (retry, sleep, timeout, etc.)
JaCoCo (10 minutes)
- Lecture: How to integrate the JaCoCo code coverage application into a Jenkins pipeline
Hands-on exercise (15 minutes)
- Set up Sonar and JaCoCo integration into your pipeline
Semantic versioning (10 minutes)
- Lecture: Semantic versioning concepts; how to introduce semantic versioning into a pipeline via the use of pipeline parameters and the Gradle properties file
Remotely loading code (5 minutes)
- Lecture: Using the Remote Loader plugin to load code remotely from another source management location; how to write code that can be pulled in remotely and used in the pipeline
Assembling the war file (5 minutes)
- Lecture: An overview of the task in the Gradle build file to assemble and package the war file artifact
Hands-on exercise (10 minutes)
- Set up parameters for semantic versioning and set up the pipeline to invoke the assemble task for the war artifact
- Break (10 minutes)
Artifactory: Publishing (15 minutes)
- Lecture: The concept of publishing as it relates to Gradle and Artifactory integration with Jenkins; an overview of the Artifactory application and how to access and use it
Artifactory: Resolving (10 minutes)
- Lecture: How to retrieve an artifact from Artifactory using a script approach
Hands-on exercise (10 minutes)
- Fill in the pieces of your pipeline to publish to Artifactory; retrieve the latest version of an artifact from it
- Break (10 minutes)
Docker (10 minutes)
- Lecture: Docker basic concepts; Docker images and containers and how they can be created and linked
Jenkins 2.0 and Docker (10 minutes)
- Lecture: How Docker integrates with a Jenkins 2 pipeline; three ways to use Docker from within a pipeline—as a “cloud” (stand-alone agent), using the built-in Docker variable, and calling Docker externally via shell commands; how Docker agents can be created and used in a pipeline
Hands-on exercise (15 minutes)
- Use the pipeline to create Docker images and containers with the warfile from Artifactory
Your Instructor
Brent Laster
Brent Laster is an R&D DevOps director at a top technology company and a global trainer, author, and speaker on open source technologies. He’s also the founder and president of Tech Skills Transformations, LLC, a company dedicated to making technology understandable and usable. Throughout his career in software development and management, Brent has always made time to learn and develop both technical and leadership skills and share them with others. He believes that regardless of the topic or technology, there’s no substitute for the excitement and sense of potential that come from providing others with the knowledge they need to accomplish their goals.