Grokking Continuous Delivery

Book description

Build and use systems that safely automate software delivery from testing through release with this jargon-busting guide to continuous delivery pipelines.

In Grokking Continuous Delivery you will learn how to:

  • Design effective CD pipelines for new and legacy projects
  • Keep your software projects release-ready
  • Maintain effective tests
  • Scale CD across multiple applications
  • Ensure pipelines give the right signals at the right time
  • Use version control as the source of truth
  • Safely automate deployments with metrics
  • Describe CD in a way that makes sense to your colleagues

Grokking Continuous Delivery teaches you the design and purpose of continuous delivery systems that you can use with any language or stack. You’ll learn directly from your mentor Christie Wilson, Google engineer and co-creator of the Tekton CI/CD framework. Using crystal-clear, well-illustrated examples, Christie lays out the practical nuts and bolts of continuous delivery for developers and pipeline designers. In each chapter, you’ll uncover the proper approaches to solve the real-world challenges of setting up a CD pipeline. With this book as your roadmap, you’ll have a clear plan for bringing CD to your team without the need for costly trial-and-error experimentation.

About the Technology
Keep your codebase release-ready. A continuous delivery pipeline automates version control, testing, and deployment with minimal developer intervention. Master the tools and practices of continuous delivery, and you’ll be able to add features and push updates quickly and consistently.

About the Book
Grokking Continuous Delivery is a friendly guide to setting up and working with a continuous delivery pipeline. Each chapter takes on a different scenario you’ll face when setting up a CD system, with real-world examples like automated scaling and testing legacy applications. Taking a tool-agnostic approach, author Christie Wilson guides you each step of the way with illustrations, crystal-clear explanations, and practical exercises to lock in what you’re learning.

What's Inside
  • Design effective CD pipelines for new and legacy projects
  • Ensure your pipelines give the right signals at the right times
  • Version control as the source of truth
  • Safely automate deployments


About the Reader
For software engineers who want to add CD to their development process.

About the Author
Christie Wilson is a software engineer at Google, where she co-created Tekton, a cloud-native CI/CD platform built on Kubernetes.

Quotes
A breath of fresh air in a complex space.
- From the Foreword by Eric Brewer, Google

This book will drive a better understanding of how to implement a modern delivery process.
- From the Foreword by Jez Humble, co-author of Continuous Delivery

The most comprehensive content I have seen on continuous delivery. By far the best practical introduction.
- William Jamir Silva, Adjust

Essential for any developer providing quality software. It goes directly to your brain.
- Daniel Vásquez, Wizeline

For anyone looking for smooth continuous integration delivery of software in real life.
- Prabhuti Prakash, Calsoft

Publisher resources

View/Submit Errata

Table of contents

  1. inside front cover
  2. Grokking Continuous Delivery
  3. Copyright
  4. dedication
  5. brief contents
  6. contents
  7. front matter
    1. forewords
    2. preface
    3. acknowledgments
    4. about this book
      1. Who should read this book
      2. How this book is organized: A road map
      3. liveBook discussion forum
    5. About the author
  8. Part 1. Introducing continuous delivery
  9. 1 Welcome to Grokking Continuous Delivery
    1. Do you need continuous delivery?
    2. Why continuous delivery?
    3. Continuous delivery
    4. Integration
    5. Continuous integration
    6. What do we deliver?
    7. Delivery
    8. Continuous delivery/deployment
    9. Elements of continuous delivery
    10. Conclusion
    11. Summary
    12. Up next . . .
  10. 2 A basic pipeline
    1. Cat Picture Website
    2. Cat Picture Website source code
    3. Cat Picture Website pipelines
    4. What’s a pipeline? What’s a task?
    5. The basic tasks in a CD pipeline
    6. Gates and transformations
    7. CD: Gates and transformations
    8. Cat Picture Website service pipeline
    9. Running the pipeline
    10. Running once a day
    11. Trying continuous integration
    12. Using notifications
    13. Scaling manual effort
    14. Automation with webhooks
    15. Scaling with webhooks
    16. Don’t push changes when broken
    17. Cat Picture Website CD
    18. What’s in a name?
    19. Conclusion
    20. Summary
    21. Up next . . .
  11. Part 2. Keeping software in a deliverable state at all times
  12. 3 Version control is the only way to roll
    1. Sasha and Sarah’s start-up
    2. All kinds of data
    3. Source and software
    4. Repositories and versions
    5. Continuous delivery and version control
    6. Git and GitHub
    7. An initial commit—with a bug!
    8. Breaking main
    9. Pushing and pulling
    10. Are we doing continuous delivery?
    11. Keep version control releasable
    12. Trigger on changes to version control
    13. Triggering the User service pipeline
    14. Building the User service
    15. The User service in the cloud
    16. Connecting to the RandomCloud database
    17. Managing the User service
    18. The User service outage
    19. Outsmarted by automation
    20. What’s the source of truth?
    21. User service config as code
    22. Configuring Deployaker
    23. Config as code
    24. Rolling out software and config changes
    25. Conclusion
    26. Summary
    27. Up next . . .
  13. 4 Use linting effectively
    1. Becky and Super Game Console
    2. Linting to the rescue!
    3. The lowdown on linting
    4. The tale of Pylint and many, many issues
    5. Legacy code: Using a systematic approach
    6. Step 1: Configure against coding standards
    7. Step 2: Establish a baseline
    8. Step 3: Enforce at submission time
    9. Adding enforcement to the pipeline
    10. Step 4: Divide and conquer
    11. Isolation: Not everything should be fixed
    12. Enforcing isolation
    13. Not all problems are created equal
    14. Types of linting issues
    15. Bugs first, style later
    16. Jumping through the hoops
    17. Conclusion
    18. Summary
    19. Up next . . .
  14. 5 Dealing with noisy tests
    1. Continuous delivery and tests
    2. Ice Cream for All outage
    3. Signal vs. noise
    4. Noisy successes
    5. How failures become noise
    6. Going from noise to signal
    7. Getting to green
    8. Another outage!
    9. Passing tests can be noisy
    10. Fixing test failures
    11. Ways of failing: Flakes
    12. Reacting to failures
    13. Fixing the test: Change the code or the test?
    14. The dangers of retries
    15. Retrying revisited
    16. Why do we retry?
    17. Get to green and stay green
    18. Conclusion
    19. Summary
    20. Up next . . .
  15. 6 Speeding up slow test suites
    1. Dog Picture Website
    2. When simple is too simple
    3. New engineer tries to submit code
    4. Tests and continuous delivery
    5. Diagnosis: Too slow
    6. The test pyramid
    7. Fast tests first
    8. Two pipelines
    9. Getting the right balance
    10. Changing the pyramid
    11. Safely adjusting tests
    12. Test Coverage
    13. Enforcing test coverage
    14. Test coverage in the pipeline
    15. Moving tests in the pyramid with coverage
    16. What to move down the pyramid?
    17. Legacy tests and FUD
    18. Running tests in parallel
    19. When can tests run in parallel?
    20. Updating the pipelines
    21. Still too slow!
    22. Test sharding, aka parallel++
    23. How to shard
    24. More complex sharding
    25. Sharded pipeline
    26. Sharding the browser tests
    27. Sharding in the pipeline
    28. Dog Picture Website’s pipelines
    29. Conclusion
    30. Summary
    31. Up next . . .
  16. 7 Give the right signals at the right times
    1. CoinExCompare
    2. Life cycle of a change
    3. CI only before merge
    4. Timeline of a change’s bugs
    5. CI only before merging misses bugs
    6. A tale of two graphs: Default to seven days
    7. A tale of two graphs: Default to 30 days
    8. Conflicts aren’t always caught
    9. What about the unit tests?
    10. PR triggering still lets bugs sneak in
    11. CI before AND after merge
    12. Option 1: Run CI periodically
    13. Option 1: Setting up periodic CI
    14. Option 2: Require branches to be up-to-date
    15. Option 2: At what cost?
    16. Option 3: Automated merge CI
    17. Option 3: Running CI with the latest main
    18. Option 3: Merge events
    19. Option 3: Merge queues
    20. Option 3: Merge queue for CoinExCompare
    21. Where can bugs still happen?
    22. Flakes and PR-triggered CI
    23. Catching flakes with periodic tests
    24. Bugs and building
    25. CI vs. build and deploy
    26. Build and deploy with the same logic
    27. Improved CI pipeline with building
    28. Timeline of a change revisited
    29. Conclusion
    30. Summary
    31. Up next . . .
  17. Part 3. Making delivery easy
  18. 8 Easy delivery starts with version control
    1. Meanwhile at Watch Me Watch
    2. The DORA metrics
    3. Velocity at Watch Me Watch
    4. Lead time for changes
    5. Watch Me Watch and elite performers
    6. Increasing velocity at Watch Me Watch
    7. Integrating with AllCatsAllTheTime
    8. Incremental feature delivery
    9. Committing skipped tests
    10. Code review and “incomplete” code
    11. Keeping up the momentum
    12. Committing work-in-progress code
    13. Reviewing work-in-progress code
    14. Meanwhile, back at the end-to-end tests
    15. Seeing the benefits
    16. Decreasing lead time for changes
    17. Continuing AllCatsAllTheTime
    18. Deployment windows and code freezes
    19. Increased velocity
    20. Conclusion
    21. Summary
    22. Up next . . .
  19. 9 Building securely and reliably
    1. Top Dog Maps
    2. When the build process is a doc
    3. Attributes of secure and reliable builds
    4. Always releasable
    5. Automated builds
    6. Build as code
    7. Use a CD service
    8. Ephemeral build environments
    9. Miguel’s plan
    10. From a doc to a script in version control
    11. Automated containerized builds
    12. Secure and reliable build process
    13. Interface changes and bugs
    14. When builds cause bugs
    15. Builds and communication
    16. Semantic versioning
    17. The importance of being versioned
    18. Another outage!
    19. Build-time dependency bugs
    20. Pinning dependencies
    21. Version pinning alone isn’t a guarantee
    22. Pinning to hashes
    23. Conclusion
    24. Summary
    25. Up next . . .
  20. 10 Deploying confidently
    1. Plenty of deployment woes
    2. DORA metrics for stability
    3. DORA metrics at Plenty of Woofs
    4. Deploying less frequently?
    5. Deploying more frequently?
    6. Daily deployments vs. outages
    7. Steps toward increasing frequency
    8. Fixing problems with the process
    9. Rolling updates
    10. Fixing a bug with a rolling update
    11. Rollbacks
    12. Rollback strategy = immediate improvement
    13. Rollback policy in action
    14. Blue-green deployments
    15. Faster time to restore with blue-green
    16. Faster and more stable with canaries
    17. Requirements for canary deployments
    18. Canary deployments with baselines
    19. Time to restore with canary deployments
    20. Increasing deployment frequency
    21. DORA metrics with daily canary deployments
    22. Continuous deployment
    23. When to use continuous deployment
    24. Mandatory QA phases
    25. QA and continuous deployment
    26. Elite DORA performance
    27. Conclusion
    28. Summary
    29. Up next . . .
  21. Part 4. CD design
  22. 11 Starter packs: From zero to CD
    1. Starter packs: Overview
    2. Recap: Universal CD pipeline tasks
    3. Prototypical release pipeline
    4. Prototypical CI pipeline
    5. Both pipelines with triggering
    6. Greenfield: Getting to CD
    7. Gulpy
    8. Greenfield: Zero to CD
    9. First step: Does it build?
    10. Picking a CD system
    11. Setting up the initial automation
    12. State of the code: Linting
    13. State of the code: Unit tests
    14. State of the code: Coverage
    15. Moving past CI: Publishing
    16. Deployment
    17. Expanding testing
    18. Tasks for integration and end-to-end tests
    19. Completing the CI pipeline
    20. Gulpy’s complete pipelines
    21. Legacy: Getting to CD
    22. Rebellious Hamster
    23. First step: Prioritize incremental goals
    24. Focusing on the pain first
    25. Pain at Rebellious Hamster
    26. Know when things are broken
    27. Isolate and add tests
    28. Legacy pipeline with more tests
    29. Make deployments more automated
    30. Creating a release pipeline
    31. Rebellious Hamster’s release pipeline
    32. Rebellious Hamster’s complete pipelines
    33. Conclusion
    34. Summary
    35. Up next . . .
  23. 12 Scripts are code, too
    1. Purrfect Bank
    2. CD problems
    3. Purrfect Bank CD overview
    4. Payment Org’s bash libraries
    5. Transaction service pipeline
    6. Evolving from one big script
    7. Principles of well-designed tasks
    8. Breaking up the giant task
    9. Updated Transaction service pipeline
    10. Debugging bash libraries
    11. Investigating the bash library bug
    12. Why was the bug introduced?
    13. What is bash for?
    14. When is bash less good?
    15. Shell scripting vs. general-purpose languages
    16. Shell script to general-purpose language
    17. Migration plan
    18. From bash library to task with bash
    19. Reusable bash inside a task
    20. From bash to Python
    21. Tasks as code
    22. CD scripts are code, too
    23. Conclusion
    24. Summary
    25. Up next . . .
  24. 13 Pipeline design
    1. PetMatch
    2. Matchmaking CD pipelines
    3. CD pipeline problems
    4. End-to-end test pipeline
    5. End-to-end test pipeline and errors
    6. Finally behavior
    7. Finally as a graph
    8. Finally in the matchmaking pipeline
    9. End-to-end test pipeline and speed
    10. Parallel execution of tasks
    11. End-to-end test pipeline and test speed
    12. Parallel execution and test sharding
    13. End-to-end test pipeline with sharding
    14. End-to-end test pipeline and signal
    15. One CI pipeline
    16. Release pipeline and signal
    17. Differences in CI
    18. Combining pipelines
    19. Release pipeline
    20. Hardcoding in the release pipeline
    21. Pipeline reuse with parameterization
    22. Using reusable pipelines
    23. Updated pipelines
    24. Solving PetMatch’s CD problems
    25. CD features to look for
    26. Conclusion
    27. Summary
    28. Up next . . .
  25. Appendices
  26. Appendix A. CD systems
    1. CD System features by chapter
    2. Feature list
    3. Argo Workflows
    4. CircleCI
    5. GitHub Actions
    6. Google Cloud Build
    7. Jenkins Pipeline
    8. Tekton
  27. Appendix B. Version control systems
    1. Version control systems
    2. Hosted version control
    3. Feature list
    4. Bitbucket
    5. GitHub
    6. GitLab
  28. index
  29. inside back cover

Product information

  • Title: Grokking Continuous Delivery
  • Author(s): Christie Wilson
  • Release date: October 2022
  • Publisher(s): Manning Publications
  • ISBN: 9781617298257