A Practical Guide to Continuous Delivery

A Practical Guide to Continuous Delivery

by Eberhard Wolff
Released February 2017
Publisher(s): Addison-Wesley Professional
ISBN: 9780134691626

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Using Continuous Delivery, you can bring software into production more rapidly, with greater reliability. A Practical Guide to Continuous Delivery is a 100% practical guide to building Continuous Delivery pipelines that automate rollouts, improve reproducibility, and dramatically reduce risk.

Eberhard Wolff introduces a proven Continuous Delivery technology stack, including Docker, Chef, Vagrant, Jenkins, Graphite, the ELK stack, JBehave, and Gatling. He guides you through applying these technologies throughout build, continuous integration, load testing, acceptance testing, and monitoring. Wolff’s start-to-finish example projects offer the basis for your own experimentation, pilot programs, and full-fledged deployments.

A Practical Guide to Continuous Delivery is for everyone who wants to introduce Continuous Delivery, with or without DevOps. For managers, it introduces core processes, requirements, benefits, and technical consequences. Developers, administrators, and architects will gain essential skills for implementing and managing pipelines, and for integrating Continuous Delivery smoothly into software architectures and IT organizations.

  • Understand the problems that Continuous Delivery solves, and how it solves them

  • Establish an infrastructure for maximum software automation

  • Leverage virtualization and Platform as a Service (PAAS) cloud solutions

  • Implement build automation and continuous integration with Gradle, Maven, and Jenkins

  • Perform static code reviews with SonarQube and repositories to store build artifacts

  • Establish automated GUI and textual acceptance testing with behavior-driven design

  • Ensure appropriate performance via capacity testing

  • Check new features and problems with exploratory testing

  • Minimize risk throughout automated production software rollouts

  • Gather and analyze metrics and logs with Elasticsearch, Logstash, Kibana (ELK), and Graphite

  • Manage the introduction of Continuous Delivery into your enterprise

  • Architect software to facilitate Continuous Delivery of new capabilities

    •

    Table of contents

    1. About This E-Book
    2. Title Page
    3. Copyright Page
    4. Dedication Page
    5. Contents at a Glance
    6. Contents
    7. Preface
    8. Acknowledgments
    9. Part I: Foundations
      1. Chapter 1: Continuous Delivery: What and How?
        1. 1.1 Introduction: What Is Continuous Delivery?
        2. 1.2 Why Software Releases are So Complicated
          1. 1.2.1 Continuous Integration Creates Hope
          2. 1.2.2 Slow and Risky Processes
          3. 1.2.3 It’s Possible to be Fast
        3. 1.3 Values of Continuous Delivery
          1. 1.3.1 Regularity
          2. 1.3.2 Traceability/Confirmability
          3. 1.3.3 Regression
        4. 1.4 Benefits of Continuous Delivery
          1. 1.4.1 Continuous Delivery for Time to Market
          2. 1.4.2 One Example
          3. 1.4.3 Implementing a Feature and Bringing It into Production
          4. 1.4.4 On to the Next Feature
          5. 1.4.5 Continuous Delivery Generates Competitive Advantages
          6. 1.4.6 Without Continuous Delivery
          7. 1.4.7 Continuous Delivery and Lean Startup
          8. 1.4.8 Effects on the Development Process
          9. 1.4.9 Continuous Delivery to Minimize Risk
          10. 1.4.10 Faster Feedback and Lean
        5. 1.5 Generations and Structure of a Continuous Delivery Pipeline
          1. 1.5.1 The Example
        6. 1.6 Conclusion
        7. Endnotes
      2. Chapter 2: Providing Infrastructure
        1. 2.1 Introduction
          1. 2.1.1 Infrastructure Automation: An Example
        2. 2.2 Installation Scripts
          1. 2.2.1 Problems of Classical Installation Scripts
        3. 2.3 Chef
          1. 2.3.1 Chef versus Puppet
          2. 2.3.2 Other Alternatives
          3. 2.3.3 Technical Foundations
          4. 2.3.4 Chef Solo
          5. 2.3.5 Chef Solo: Conclusion
          6. 2.3.6 Knife and Chef Server
          7. 2.3.7 Chef Server: Conclusion
        4. 2.4 Vagrant
          1. 2.4.1 An Example with Chef and Vagrant
          2. 2.4.2 Vagrant: Conclusion
        5. 2.5 Docker
          1. 2.5.1 Docker’s Solution
          2. 2.5.2 Creating Docker Containers
          3. 2.5.3 Running the Example Application with Docker
          4. 2.5.4 Docker and Vagrant
          5. 2.5.5 Docker Machine
          6. 2.5.6 Complex Configurations with Docker
          7. 2.5.7 Docker Compose
        6. 2.6 Immutable Server
          1. 2.6.1 Disadvantages of Idempotency
          2. 2.6.2 Immutable Server and Docker
        7. 2.7 Infrastructure as Code
          1. 2.7.1 Testing Infrastructure as Code
        8. 2.8 Platform as a Service (PaaS)
        9. 2.9 Handling Data and Databases
          1. 2.9.1 Handling Schemas
          2. 2.9.2 Test and Master Data
        10. 2.10 Conclusion
        11. Endnotes
    10. Part II: The Continuous Delivery Pipeline
      1. Chapter 3: Build Automation and Continuous Integration
        1. 3.1 Introduction
          1. 3.1.1 Build Automation: An Example
        2. 3.2 Build Automation and Build Tools
          1. 3.2.1 Build Tools in the Java World
          2. 3.2.2 Ant
          3. 3.2.3 Maven
          4. 3.2.4 Gradle
          5. 3.2.5 Additional Build Tools
          6. 3.2.6 Choosing the Right Tool
        3. 3.3 Unit Tests
          1. 3.3.1 Writing Good Unit Tests
          2. 3.3.2 TDD—Test-Driven Development
          3. 3.3.3 Clean Code and Software Craftsmanship
        4. 3.4 Continuous Integration
          1. 3.4.1 Jenkins
          2. 3.4.2 Continuous Integration Infrastructure
          3. 3.4.3 Conclusion
        5. 3.5 Measuring Code Quality
          1. 3.5.1 SonarQube
          2. 3.6 Managing Artifacts
          3. 3.6.1 Integration into the Build
          4. 3.6.2 Advanced Features of Repositories
        6. 3.7 Conclusion
        7. Endnotes
      2. Chapter 4: Acceptance Tests
        1. 4.1 Introduction
          1. 4.1.1 Acceptance Tests: An Example
        2. 4.2 The Test Pyramid
        3. 4.3 What Are Acceptance Tests?
          1. 4.3.1 Automated Acceptance Tests
          2. 4.3.2 More Than Just an Increase in Efficiency
          3. 4.3.3 Manual Tests
          4. 4.3.4 What about the Customer?
          5. 4.3.5 Acceptance versus Unit Tests
          6. 4.3.6 Test Environments
        4. 4.4 GUI-Based Acceptance Tests
          1. 4.4.1 Problems of GUI Tests
          2. 4.4.2 Abstractions against Fragile GUI Tests
          3. 4.4.3 Automation with Selenium
          4. 4.4.4 Web Driver API
          5. 4.4.5 Tests without Web Browser: HtmlUnit
          6. 4.4.6 Selenium Web Driver API
          7. 4.4.7 Selenium IDE
          8. 4.4.8 Problems with Automated GUI Tests
          9. 4.4.9 Executing GUI Tests
          10. 4.4.10 Exporting the Tests as Code
          11. 4.4.11 Manual Modifications of the Test Cases
          12. 4.4.12 Test Data
          13. 4.4.13 Page Object
        5. 4.5 Alternative Tools for GUI Tests
          1. 4.5.1 PhantomJS
          2. 4.5.2 Windmill
        6. 4.6 Textual Acceptance Tests
          1. 4.6.1 Behavior-Driven Development
          2. 4.6.2 Different Adaptors
        7. 4.7 Alternative Frameworks
        8. 4.8 Strategies for Acceptance Tests
          1. 4.8.1 The Right Tool
          2. 4.8.2 Rapid Feedback
          3. 4.8.3 Test Coverage
        9. 4.9 Conclusion
        10. Endnotes
      3. Chapter 5: Capacity Tests
        1. 5.1 Introduction
          1. 5.1.1 Capacity Tests: An Example
        2. 5.2 Capacity Tests—How?
          1. 5.2.1 Objectives of Capacity Tests
          2. 5.2.2 Data Volumes and Environments
          3. 5.2.3 Performance Tests Only at the End of the Implementation?
          4. 5.2.4 Capacity Tests = Risk Management
          5. 5.2.5 Simulating Users
          6. 5.2.6 Documenting Performance Requirements
          7. 5.2.7 Hardware for Capacity Tests
          8. 5.2.8 Cloud and Virtualization
          9. 5.2.9 Minimizing Risk by Continuous Testing
          10. 5.2.10 Capacity Tests—Sensible or Not?
        3. 5.3 Implementing Capacity Tests
        4. 5.4 Capacity Tests with Gatling
          1. 5.4.1 Demo versus Real Life
        5. 5.5 Alternatives to Gatling
          1. 5.5.1 Grinder
          2. 5.5.2 Apache JMeter
          3. 5.5.3 Tsung
          4. 5.5.4 Commercial Solutions
        6. 5.6 Conclusion
        7. Endnotes
      4. Chapter 6: Exploratory Testing
        1. 6.1 Introduction
          1. 6.1.1 Exploratory Tests: An Example
        2. 6.2 Why Exploratory Tests?
          1. 6.2.1 Sometimes Manual Testing Is Still Better
          2. 6.2.2 Test by the Customers
          3. 6.2.3 Manual Tests for Non-Functional Requirements
        3. 6.3 How to Go About It?
          1. 6.3.1 Missions Guide the Tests
          2. 6.3.2 Automated Environment
          3. 6.3.3 Showcases as a Basis
          4. 6.3.4 Example: An E-Commerce Application
          5. 6.3.5 Beta Tests
          6. 6.3.6 Session-Based Tests
        4. 6.4 Conclusion
        5. Endnotes
      5. Chapter 7: Deploy—The Rollout in Production
        1. 7.1 Introduction
          1. 7.1.1 Deployment: An Example
        2. 7.2 Rollout and Rollback
          1. 7.2.1 Benefits
          2. 7.2.2 Disadvantages
        3. 7.3 Roll Forward
          1. 7.3.1 Benefits
          2. 7.3.2 Disadvantages
        4. 7.4 Blue/Green Deployment
          1. 7.4.1 Benefits
          2. 7.4.2 Disadvantages
        5. 7.5 Canary Releasing
          1. 7.5.1 Benefits
          2. 7.5.2 Disadvantages
        6. 7.6 Continuous Deployment
          1. 7.6.1 Benefits
          2. 7.6.2 Disadvantages
        7. 7.7 Virtualization
          1. 7.7.1 Physical Hosts
        8. 7.8 Beyond Web Applications
        9. 7.9 Conclusion
        10. Endnotes
      6. Chapter 8: Operations
        1. 8.1 Introduction
          1. 8.1.1 Operate—An Example
        2. 8.2 Challenges in Operations
        3. 8.3 Log Files
          1. 8.3.1 What Should Be Logged?
          2. 8.3.2 Tools for Processing Log Files
          3. 8.3.3 Logging in the Example Application
        4. 8.4 Analyzing Logs of the Example Application
          1. 8.4.1 Analyses with Kibana
          2. 8.4.2 ELK—Scalability
        5. 8.5 Other Technologies for Logs
        6. 8.6 Advanced Log Techniques
          1. 8.6.1 Anonymization
          2. 8.6.2 Performance
          3. 8.6.3 Time
          4. 8.6.4 Ops Database
        7. 8.7 Monitoring
        8. 8.8 Metrics with Graphite
        9. 8.9 Metrics in the Example Application
          1. 8.9.1 Structure of the Example
        10. 8.10 Other Monitoring Solutions
        11. 8.11 Additional Challenges When Operating an Application
          1. 8.11.1 Scripts
          2. 8.11.2 Applications in a Client’s Data Center
        12. 8.12 Conclusion
        13. Endnotes
    11. Part III: Management, Organization, and Architecture for Continuous Delivery
      1. Chapter 9: Introducing Continuous Delivery into Your Enterprise
        1. 9.1 Introduction
        2. 9.2 Continuous Delivery Right from the Start
        3. 9.3 Value Stream Mapping
          1. 9.3.1 Value Stream Mapping Describes the Sequence of Events
          2. 9.3.2 Optimizations
        4. 9.4 Additional Measures for Optimization
          1. 9.4.1 Quality Investments
          2. 9.4.2 Costs
          3. 9.4.3 Benefits
          4. 9.4.4 Do not Check in on a Red Build!
          5. 9.4.5 Stop the Line
          6. 9.4.6 5 Whys
          7. 9.4.7 DevOps
        5. 9.5 Conclusion
        6. Endnotes
      2. Chapter 10: Continuous Delivery and DevOps
        1. 10.1 Introduction
        2. 10.2 What Is DevOps?
          1. 10.2.1 Problems
          2. 10.2.2 The Client Perspective
          3. 10.2.3 Pioneer: Amazon
          4. 10.2.4 DevOps
        3. 10.3 Continuous Delivery and DevOps
          1. 10.3.1 DevOps: More Than Continuous Delivery
          2. 10.3.2 Individual Responsibility and Self-Organization
          3. 10.3.3 Technology Decisions
          4. 10.3.4 Less Central Control
          5. 10.3.5 Technology Pluralism
          6. 10.3.6 Exchange Between Teams
          7. 10.3.7 Architecture
        4. 10.4 Continuous Delivery without DevOps?
          1. 10.4.1 Terminating the Continuous Delivery Pipeline
        5. 10.5 Conclusion
        6. Endnotes
      3. Chapter 11: Continuous Delivery, DevOps, and Software Architecture
        1. 11.1 Introduction
        2. 11.2 Software Architecture
          1. 11.2.1 Why Software Architecture?
        3. 11.3 Optimizing Architecture for Continuous Delivery
          1. 11.3.1 Smaller Deployment Units
        4. 11.4 Interfaces
          1. 11.4.1 Postel’s Law or the Robustness Principle
          2. 11.4.2 Design for Failure
          3. 11.4.3 State
        5. 11.5 Databases
          1. 11.5.1 Keeping Databases Stable
          2. 11.5.2 Database = Component
          3. 11.5.3 Views and Stored Procedures
          4. 11.5.4 A Database per Component
          5. 11.5.5 NoSQL Databases
        6. 11.6 Microservices
          1. 11.6.1 Microservices and Continuous Delivery
          2. 11.6.2 Introducing Continuous Delivery with Microservices
          3. 11.6.3 Microservices Entail Continuous Delivery
          4. 11.6.4 Organization
        7. 11.7 Handling New Features
          1. 11.7.1 Feature Branches
          2. 11.7.2 Feature Toggles
          3. 11.7.3 Benefits
          4. 11.7.4 Use Cases for Feature Toggles
          5. 11.7.5 Disadvantages
        8. 11.8 Conclusion
        9. Endnotes
      4. Chapter 12: Conclusion: What Are the Benefits?
        1. Endnotes
    12. Index
    13. Code Snippets

    Product information

    • Title: A Practical Guide to Continuous Delivery
    • Author(s): Eberhard Wolff
    • Release date: February 2017
    • Publisher(s): Addison-Wesley Professional
    • ISBN: 9780134691626