Automating DevOps with GitLab CI/CD Pipelines

Automating DevOps with GitLab CI/CD Pipelines

by Christopher Cowell, Nicholas Lotz, Chris Timberlake
Released February 2023
Publisher(s): Packt Publishing
ISBN: 9781803233000

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Use GitLab CI/CD pipelines for automating and deploying different steps of your software development lifecycle using best practices and troubleshooting methods.

Key Features

  • Reap the power of GitLab CI/CD pipelines at every stage of your software development lifecycle
  • Learn how GitLab makes Git easier to use and more powerful when committing and reviewing code
  • Cement your understanding using hands-on tutorials and extensive self-assessment exercises
  • Purchase of the print or Kindle book includes a free eBook in the PDF format

Book Description

Developers and release engineers understand the high stakes involved in building, packaging, and deploying code correctly. Ensuring that your code is functionally correct, fast, and secure is a time-consuming and complex task. Code implementation, development, and deployment can be conducted efficiently using GitLab CI/CD pipelines.

Automating DevOps with GitLab CI/CD Pipelines begins with the basics of Git and GitLab, showing how to commit and review code. You’ll learn to set up GitLab Runners for executing and autoscaling CI/CD pipelines and creating and configuring pipelines for many software development lifecycle steps. You'll also discover where to find pipeline results in GitLab, and how to interpret those results. Through the course of the book, you’ll become well-equipped with deploying code to different environments, advancing CI/CD pipeline features such as connecting GitLab to a Kubernetes cluster and using GitLab with Terraform, triggering pipelines and improving pipeline performance and using best practices and troubleshooting tips for uncooperative pipelines. In-text examples, use cases, and self-assessments will reinforce the important CI/CD, GitLab, and Git concepts, and help you prepare for interviews and certification exams related to GitLab.

By the end of this book, you'll be able to use GitLab to build CI/CD pipelines that automate all the DevOps steps needed to build and deploy high-quality, secure code.

What you will learn

  • Gain insights into the essentials of Git, GitLab, and DevOps
  • Understand how to create, view, and run GitLab CI/CD pipelines
  • Explore how to verify, secure, and deploy code with GitLab CI/CD pipelines
  • Configure and use GitLab Runners to execute CI/CD pipelines
  • Explore advanced GitLab CI/CD pipeline features like DAGs and conditional logic
  • Follow best practices and troubleshooting methods of GitLab CI/CD pipelines
  • Implement end-to-end software development lifecycle workflows using examples

Who this book is for

This book is for DevOps/DevSecOps engineers, application developers, release engineers, quality assurance engineers, security engineers, SREs, and sysadmins looking to implement fast, secure and automated software development lifecycle tasks using continuous integration and continuous delivery (CI/CD) pipelines in GitLab. Basic knowledge of major stages of the software development life cycle and DevOps processes will be helpful.

Table of contents

  1. Automating DevOps with GitLab CI/CD Pipelines
  2. Contributors
  3. About the authors
  4. About the reviewers
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Conventions used
    5. Get in touch
    6. Share Your Thoughts
    7. Download a free PDF copy of this book
  6. Part 1 Getting Started with DevOps, Git, and GitLab
  7. Chapter 1: Understanding Life Before DevOps
    1. Introducing the Hats for Cats web app
    2. Building and verifying code manually
      1. Building code manually
      2. Verifying code manually
      3. More challenges of verifying code
    3. Security-testing code manually
      1. Static code analysis
      2. Secret detection
      3. Dynamic analysis
      4. Dependency scanning
      5. Container scanning
      6. Manual security testing summary
    4. Packaging and deploying code manually
      1. License compliance scanning
      2. Deploying software
    5. Problems with manual software development life cycle practices
    6. Solving problems with DevOps
      1. How GitLab implements DevOps
    7. Summary
  8. Chapter 2: Practicing Basic Git Commands
    1. Technical requirements
    2. Why use Git?
      1. What is a version control system?
      2. What problems does a VCS solve?
      3. Why Git is popular
      4. Drawbacks of Git
    3. Committing code to keep it safe
      1. Excluding files from a repository
    4. Tagging commits to identify versions of code
    5. Branching code for developing in an isolated space
      1. Git commands for managing branches
      2. Handling merge conflicts
    6. Syncing local and remote copies of repositories
      1. The “golden” repository
      2. Configuring remote repositories
      3. Pushing
      4. Fetching
      5. Pulling
    7. Additional resources for learning Git
    8. Summary
  9. Chapter 3: Understanding GitLab Components
    1. Technical requirements
    2. Emphasizing the “why” over the “how”
    3. Introducing the GitLab platform
      1. What is GitLab?
      2. What problem does GitLab solve?
      3. The verify, secure, and release stages
    4. Organizing work into projects and groups
      1. Example – organizing your Hats for Cats work
    5. Tracking work with issues
      1. The structure of a GitLab issue
      2. The kinds of tasks that issues can represent
      3. Labels
      4. Issue workflows
    6. Editing files safely with commits, branches, and merge requests
      1. Commit history
      2. Merging one Git branch into another
      3. The three amigos – issues, branches, and merge requests
      4. When two amigos are enough
      5. How are issues and merge requests different?
    7. Enabling DevOps practices with GitLab flow
    8. Summary
  10. Chapter 4: Understanding GitLab’s CI/CD Pipeline Structure
    1. Technical requirements
    2. Defining the terms pipeline, CI, and CD
      1. Understanding what a pipeline is
      2. Defining one pipeline per project
      3. Understanding different uses of the term “pipeline”
      4. Viewing a list of pipelines
      5. CI – finding out if your code is good
      6. CD – finding out where your code should go (and putting it there)
      7. GitLab Runners
    3. Parts of a pipeline – stages, jobs, and commands
      1. Stages
      2. Jobs
      3. Commands
      4. Fitting the pipeline pieces together
    4. Running GitLab CI/CD pipelines
      1. Branch pipelines
      2. Git tag pipelines
      3. Other types of pipelines
      4. Skipping pipelines
    5. Reading GitLab CI/CD pipeline statuses
    6. Configuring GitLab CI/CD pipelines
    7. Summary
  11. Part 2 Automating DevOps Stages with GitLab CI/CD Pipelines
  12. Chapter 5: Installing and Configuring GitLab Runners
    1. Technical requirements
    2. Defining GitLab runners and their relationship to CI/CD
      1. GitLab Runner is an open source application written in Go
      2. GitLab Runner runs CI/CD jobs specified in .gitlab-ci.yml
    3. The runner architecture and supported platforms
      1. GitLab Runner is supported on most platforms and architectures
      2. Runners can be specific, group, or shared
      3. Each runner has a defined executor
      4. Runner tags restrict which runners can pick up which jobs
    4. Installing the Runner agent
      1. Installing GitLab Runner
      2. Registering a runner with GitLab
    5. Considerations regarding the various runner types and executors
      1. Performance considerations
      2. Security considerations
      3. Monitoring considerations
    6. Summary
  13. Chapter 6: Verifying Your Code
    1. Technical requirements
    2. Building code in a CI/CD pipeline
      1. Compiling Java with javac
      2. Compiling Java with Maven
      3. Compiling C with Gnu Compiler Collection (GCC)
      4. Storing built code as artifacts
    3. Checking code quality in a CI/CD pipeline
      1. Enabling Code Quality
      2. Viewing Code Quality results
    4. Running automated functional tests in a CI/CD pipeline
      1. Enabling automated functional tests
      2. Viewing automated functional test results
    5. Fuzz testing in a CI/CD pipeline
      1. The architecture and workflow of fuzz testing
      2. A fuzz testing workflow
      3. Viewing the results of fuzz testing
      4. Extra considerations when fuzz testing
      5. Fuzz testing with a corpus
    6. Checking accessibility in a CI/CD pipeline
      1. Enabling accessibility testing
      2. Viewing accessibility testing results
    7. Additional ways to verify your code
      1. Code coverage
      2. Browser performance testing
      3. Load performance testing
    8. Summary
  14. Chapter 7: Securing Your Code
    1. Technical requirements
    2. Understanding GitLab’s security scanning strategy
      1. GitLab uses open-source scanners
      2. Scanners are packaged as Docker images
      3. Some scanners use different analyzers for different languages
      4. Vulnerabilities don’t stop the pipeline
      5. Findings appear in three different reports
      6. Pipelines can use non-GitLab-provided scanners
    3. Using SAST to scan your source code for vulnerabilities
      1. Understanding SAST
      2. Enabling SAST
      3. Configuring SAST
      4. Viewing SAST’s findings
    4. Using Secret Detection to find private information in your repository
      1. Understanding Secret Detection
      2. Enabling and configuring Secret Detection
      3. Viewing Secret Detection’s findings
    5. Using DAST to find vulnerabilities in web applications
      1. Understanding DAST
      2. Enabling and configuring DAST
      3. Viewing DAST’s findings
    6. Using Dependency Scanning to find vulnerabilities in dependencies
      1. Understanding Dependency Scanning
      2. Enabling and configuring Dependency Scanning
      3. Viewing Dependency Scanning’s findings
    7. Using Container Scanning to find vulnerabilities in Docker images
      1. Understanding Container Scanning
      2. Enabling and configuring Container Scanning
      3. Viewing Container Scanning’s findings
    8. Using License Compliance to manage licenses of dependencies
      1. Understanding License Compliance
      2. Enabling and configuring License Compliance
      3. Viewing License Compliance’s findings
    9. Using IaC Scanning to find problems in infrastructure configuration files
      1. Understanding IaC Scanning
      2. Enabling and configuring IaC scanning
      3. Viewing IaC Scanning’s findings
    10. Understanding the different types of security reports
    11. Managing security vulnerabilities
    12. Integrating outside security scanners
    13. Summary
  15. Chapter 8: Packaging and Deploying Code
    1. Technical requirements
    2. Storing code in GitLab’s package registry for later re-use
      1. Locating GitLab’s container and package registries
      2. Getting started with the package registry
      3. Supported package formats
      4. Authenticating to the registry
      5. Building and pushing packages to the package registry
      6. Building and pushing packages to the container registry
    3. Storing code in GitLab’s container and package registries for later deployment
      1. Using images from the container registry
      2. Using packages from the package registry
    4. Deploying to different environments using GitLab Flow
    5. Deploying to a review app for testing
    6. Deploying to real-world production environments
    7. Deploying to a Kubernetes cluster
      1. The CI/CD workflow
      2. A GitOps workflow
    8. Summary
  16. Part 3 Next Steps for Improving Your Applications with GitLab
  17. Chapter 9: Enhancing the Speed and Maintainability of CI/CD Pipelines
    1. Accelerating pipelines with directed acyclic graphs and parent-child architecture
      1. How to create a DAG in GitLab CI
    2. Building code for multiple architectures
    3. When and how to leverage caching or artifacts
      1. Caching characteristics
      2. Artifact characteristics
      3. Using caching
      4. Using artifacts
      5. Leveraging artifacts as job dependencies
    4. Reducing repeated configuration code with anchors and extensions
      1. Anchors
      2. The extends: keyword
      3. Reference tags
    5. Improving maintainability by combining multiple pipelines and leveraging parent-child pipelines
      1. Leveraging includes for maintainability
      2. Leveraging includes for reusability
      3. Includes from remote areas
      4. Leveraging parent-child pipelines
    6. Securing and accelerating jobs with purpose-built containers
      1. A purpose-built container example
    7. Summary
  18. Chapter 10: Extending the Reach of CI/CD Pipelines
    1. Using CI/CD pipelines to spot performance problems
      1. How to integrate browser performance testing
      2. How to integrate load performance testing with k6
      3. Using feature flags to allow business-driven release decisions
      4. How to configure your application for feature flags
    2. Integrating third-party tools into your CI/CD pipelines
      1. Creating our tool container’s Dockerfile
      2. Automating our container’s build
      3. Container scanning
      4. Invoking the third-party tool
    3. Using CI/CD pipelines for developing mobile apps
      1. Requirements
      2. Fastlane
      3. Fastlane – deployment
      4. Fastlane – automated testing
    4. Summary
  19. Chapter 11: End-to-End Example
    1. Technical requirements
    2. Setting up your environment
      1. Making a GitLab project
      2. Planning work with GitLab issues
      3. Setting up a local Git repository
    3. Writing code
      1. Creating a Git branch to work on
      2. Creating an MR
      3. Committing and pushing code
    4. Establishing the pipeline infrastructure
      1. Creating a pipeline
      2. Creating a runner
    5. Verifying your code
      1. Adding functional tests to the pipeline
      2. Adding Code Quality scanning to the pipeline
      3. Adding a fuzz test to the pipeline
    6. Securing your code
      1. Adding SAST to the pipeline
      2. Adding Secret Detection to the pipeline
      3. Adding Dependency Scanning to the pipeline
      4. Adding License Compliance to the pipeline
      5. Integrating a third-party security scanner into the pipeline
    7. Improving your pipeline
      1. Using a DAG to speed up the pipeline
      2. Breaking the pipeline into several files
    8. Delivering your code to the right environment
      1. Deploying the code
    9. Summary
  20. Chapter 12: Troubleshooting and the Road Ahead with GitLab
    1. Technical requirements
    2. Troubleshooting and best practices for common pipeline problems
      1. Troubleshooting CI/CD syntax and logic
      2. Troubleshooting pipeline operation and runner assignment
    3. Managing your operational infrastructure using GitOps
      1. Using Terraform to deploy and update infrastructure state
      2. Using Ansible to manage resource configurations
    4. Future industry trends
      1. Automation will create more software at a larger scale
      2. Abstraction will lead to everything-as-code business models
      3. Reduced cycle time will help teams release better software faster
    5. Conclusion and next steps
  21. Index
    1. Why subscribe?
  22. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Share Your Thoughts
    3. Download a free PDF copy of this book

Product information

  • Title: Automating DevOps with GitLab CI/CD Pipelines
  • Author(s): Christopher Cowell, Nicholas Lotz, Chris Timberlake
  • Release date: February 2023
  • Publisher(s): Packt Publishing
  • ISBN: 9781803233000