O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Learning DevOps: Continuously Deliver Better Software

Book Description

Learn to use some of the most exciting and powerful tools to deliver world-class quality software with continuous delivery and DevOps

About This Book

  • Get to know the background of DevOps so you understand the collaboration between different aspects of an IT organization and a software developer
  • Deploy top-quality software and ensure software maintenance and release management with this practical guide
  • This course covers some of the most exciting technology available to DevOps engineers, and demonstrates multiple techniques for using them
  • Real-world and realistic examples are provided to help you as you go about the implementation and adoption of continuous delivery and DevOps

Who This Book Is For

This course is for developers who want to understand how the infrastructure that builds today's enterprises works, and how to painlessly and regularly ship quality software.

What You Will Learn

  • Set up and familiarize yourself with all the tools you need to be efficient with DevOps
  • Design an application that is suitable for continuous deployment systems with DevOps in mind
  • Test the code using automated regression testing with Jenkins Selenium
  • Managing the lifecycle of hosts, from creation to ongoing management using Puppet Razor
  • Find out how to manage, use, and work with Code in the Git version management system
  • See what traps, pitfalls, and hurdles to look out for as you implement continuous delivery and DevOps

In Detail

Harness the power of DevOps to boost your skill set and make your IT organization perform better. If you’re keen to employ DevOps techniques to better your software development, this course contains all you need to overcome the day-to-day complications of managing complex infrastructures the DevOps way.

Start with your first module – Practical DevOps - that encompasses the entire flow from code from testing to production. Get a solid ground-level knowledge of how to monitor code for any anomalies, perform code testing, and make sure the code is running smoothly through a series of real-world exercise, and develop practical skills by creating a sample enterprise Java application.

In the second module, run through a series of tailored mini-tutorials designed to give you a complete understanding of every DevOps automation technique. Create real change in the way you deliver your projects by utilizing some of the most commendable software available today. Go from your first steps of managing code in Git to configuration management in Puppet, monitoring using Sensu, and more.

In the final module, get to grips with the continuous delivery techniques that will help you reduce the time and effort that goes into the delivery and support of software.

This Learning Path combines some of the best that Packt has to offer in one complete, curated package. It includes content from the following Packt products:

  • Practical DevOps by Joakim Verona
  • DevOps Automation Cookbook by Michael Duffy
  • Continuous Delivery and DevOps : A Quickstart Guide - Second Edition by Paul Swartout

Style and approach

This course is an easy to follow project based guide for all those with a keen interest in deploying world-class software using some of the most effective and remarkable technologies available.

Downloading the example code for this book You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Table of Contents

  1. Learning DevOps: Continuously Deliver Better Software
    1. Table of Contents
    2. Learning DevOps: Continuously Deliver Better Software
    3. Learning DevOps: Continuously Deliver Better Software
    4. Credits
    5. Preface
      1. What this learning path covers
      2. What you need for this learning path
      3. Who this learning path is for
      4. Reader feedback
      5. Customer support
        1. Downloading the example code
        2. Errata
        3. Piracy
        4. Questions
    6. I. Module 1
      1. 1. Introduction to DevOps and Continuous Delivery
        1. Introducing DevOps
        2. How fast is fast?
        3. The Agile wheel of wheels
        4. Beware the cargo cult Agile fallacy
        5. DevOps and ITIL
        6. Summary
      2. 2. A View from Orbit
        1. The DevOps process and Continuous Delivery – an overview
          1. The developers
          2. The revision control system
          3. The build server
          4. The artifact repository
          5. Package managers
          6. Test environments
          7. Staging/production
        2. Release management
        3. Scrum, Kanban, and the delivery pipeline
        4. Wrapping up – a complete example
        5. Identifying bottlenecks
        6. Summary
      3. 3. How DevOps Affects Architecture
        1. Introducing software architecture
        2. The monolithic scenario
        3. Architecture rules of thumb
        4. The separation of concerns
        5. The principle of cohesion
        6. Coupling
        7. Back to the monolithic scenario
        8. A practical example
        9. Three-tier systems
        10. The presentation tier
        11. The logic tier
        12. The data tier
        13. Handling database migrations
        14. Rolling upgrades
        15. Hello world in Liquibase
        16. The changelog file
        17. The pom.xml file
        18. Manual installation
        19. Microservices
        20. Interlude – Conway's Law
        21. How to keep service interfaces forward compatible
        22. Microservices and the data tier
        23. DevOps, architecture, and resilience
        24. Summary
      4. 4. Everything is Code
        1. The need for source code control
        2. The history of source code management
        3. Roles and code
        4. Which source code management system?
        5. A word about source code management system migrations
        6. Choosing a branching strategy
        7. Branching problem areas
        8. Artifact version naming
        9. Choosing a client
        10. Setting up a basic Git server
        11. Shared authentication
        12. Hosted Git servers
        13. Large binary files
        14. Trying out different Git server implementations
        15. Docker intermission
        16. Gerrit
          1. Installing the git-review package
          2. The value of history revisionism
        17. The pull request model
        18. GitLab
        19. Summary
      5. 5. Building the Code
        1. Why do we build code?
        2. The many faces of build systems
        3. The Jenkins build server
        4. Managing build dependencies
        5. The final artifact
        6. Cheating with FPM
        7. Continuous Integration
        8. Continuous Delivery
        9. Jenkins plugins
        10. The host server
        11. Build slaves
        12. Software on the host
        13. Triggers
        14. Job chaining and build pipelines
        15. A look at the Jenkins filesystem layout
        16. Build servers and infrastructure as code
          1. Building by dependency order
        17. Build phases
        18. Alternative build servers
        19. Collating quality measures
        20. About build status visualization
        21. Taking build errors seriously
        22. Robustness
        23. Summary
      6. 6. Testing the Code
        1. Manual testing
        2. Pros and cons with test automation
        3. Unit testing
        4. JUnit in general and JUnit in particular
          1. A JUnit example
        5. Mocking
        6. Test Coverage
        7. Automated integration testing
          1. Docker in automated testing
          2. Arquillian
        8. Performance testing
        9. Automated acceptance testing
        10. Automated GUI testing
        11. Integrating Selenium tests in Jenkins
        12. JavaScript testing
        13. Testing backend integration points
        14. Test-driven development
        15. REPL-driven development
        16. A complete test automation scenario
          1. Manually testing our web application
          2. Running the automated test
          3. Finding a bug
          4. Test walkthrough
          5. Handling tricky dependencies with Docker
        17. Summary
      7. 7. Deploying the Code
        1. Why are there so many deployment systems?
          1. Configuring the base OS
          2. Describing clusters
          3. Delivering packages to a system
        2. Virtualization stacks
        3. Executing code on the client
          1. A note about the exercises
        4. The Puppet master and Puppet agents
        5. Ansible
        6. PalletOps
        7. Deploying with Chef
        8. Deploying with SaltStack
        9. Salt versus Ansible versus Puppet versus PalletOps execution models
        10. Vagrant
        11. Deploying with Docker
        12. Comparison tables
        13. Cloud solutions
        14. AWS
        15. Azure
        16. Summary
      8. 8. Monitoring the Code
        1. Nagios
        2. Munin
        3. Ganglia
        4. Graphite
        5. Log handling
          1. Client-side logging libraries
          2. The ELK stack
        6. Summary
      9. 9. Issue Tracking
        1. What are issue trackers used for?
        2. Some examples of workflows and issues
        3. What do we need from an issue tracker?
        4. Problems with issue tracker proliferation
        5. All the trackers
          1. Bugzilla
          2. Trac
          3. Redmine
          4. The GitLab issue tracker
          5. Jira
        6. Summary
      10. 10. The Internet of Things and DevOps
        1. Introducing the IoT and DevOps
        2. The future of the IoT according to the market
        3. Machine-to-machine communication
        4. IoT deployment affects software architecture
        5. IoT deployment security
        6. Okay, but what about DevOps and the IoT again?
        7. A hands-on lab with an IoT device for DevOps
        8. Summary
    7. II. Module 2
      1. 1. Basic Command Line Tools
        1. Introduction
        2. Controlling network interfaces
          1. Getting ready
          2. How to do it…
          3. See also
        3. Monitoring network details with the IP command
          1. Getting ready
          2. How to do it…
        4. Monitoring connections using the ss command
          1. Getting ready
          2. How to do it…
        5. Gathering basic OS statistics
          1. Getting ready
          2. How to do it…
        6. Viewing historical resource usage with SAR
          1. Getting ready
          2. How to do it…
        7. Installing and configuring a Git client
          1. Getting ready
          2. How to do it…
        8. Creating an SSH key for Git
          1. Getting ready
          2. How to do it
          3. How it works…
        9. Using ssh-copy-id to copy keys
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        10. Creating a new Git repository
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        11. Cloning an existing Git repository
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        12. Checking changes into a Git repository
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        13. Pushing changes to a Git remote
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        14. Creating a Git branch
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
      2. 2. Ad Hoc Tasks with Ansible
        1. Introduction
        2. Installing an Ansible control node on Ubuntu
          1. Getting ready
          2. How to do it…
          3. See also
        3. Installing an Ansible control node on CentOS
          1. Getting ready
          2. How to do it…
          3. See also
        4. Creating an Ansible inventory
          1. Getting ready
          2. How to do it…
          3. See also
        5. Using the raw module to install python-simplejson
          1. Getting ready
          2. How to do it…
          3. See also
        6. Installing packages with Ansible
          1. Getting ready
          2. How to do it...
          3. See also
        7. Restarting services using Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        8. Executing freeform commands with Ansible
          1. Getting ready
          2. How to do it…
        9. Managing users with Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        10. Managing SSH keys with Ansible
          1. Getting ready
          2. How to do it...
          3. See also
      3. 3. Automatic Host builds
        1. Introduction
        2. Creating an Apt mirror using aptly
          1. Getting ready
          2. How to do it…
          3. See also
        3. Automated installation using PXE boot and a Preseed file
          1. Getting ready
          2. How to do it…
          3. See also
        4. Automating post-installation tasks
          1. Getting ready
          2. How to do it…
      4. 4. Virtualization with VMware ESXi
        1. Introduction
        2. Installing ESXi
          1. Getting ready
          2. How to do it…
        3. Installing and using the vSphere Client
          1. Getting ready
          2. How to do it….
        4. Allowing SSH access to ESXi
          1. Getting ready
          2. How to do it…
        5. Creating a new guest
          1. Getting ready
          2. How to do it...
        6. Allocating resources to a guest
          1. Getting ready
          2. How to do it…
        7. Using the ESXi command line to start, stop, and destroy guests
          1. Getting ready
          2. How to do it…
        8. Managing command-line snapshots
          1. Getting ready
          2. How to do it…
        9. Tuning the host for guest performance
          1. Getting ready
          2. How to do it…
          3. See also
      5. 5. Automation with Ansible
        1. Introduction
        2. Installing Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        3. Creating a scaffold Playbook
          1. Getting ready
          2. How to do it
        4. Creating a common role
          1. Getting ready
          2. How to do it…
          3. See also
        5. Creating a webserver using Ansible and Nginx
          1. Getting ready
          2. How to do it…
          3. See also
        6. Creating an application server role using Tomcat and Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        7. Installing MySQL using Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        8. Installing and managing HAProxy with Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        9. Using ServerSpec to test your Playbook
          1. Getting ready
          2. How to do it…
          3. See also
      6. 6. Containerization with Docker
        1. Introduction
        2. Installing Docker
          1. Getting ready
          2. How to do it…
          3. See also
        3. Pulling an image from the public Docker registry
          1. Getting ready
          2. How to do it…
          3. See also
        4. Performing basic Docker operations
          1. Getting ready
          2. How to do it…
          3. See also
        5. Running a container interactively
          1. Getting ready
          2. How to do it…
          3. See also
        6. Creating a Dockerfile
          1. Getting ready
          2. How to do it…
          3. See also
        7. Running a container in detached mode
          1. Getting ready
          2. How to do it…
          3. See also
        8. Saving and restoring a container
          1. Getting ready
          2. How to do it…
          3. See also
        9. Using the host only network
          1. Getting ready
          2. How to do it…
          3. See also
        10. Running a private Docker registry
          1. Getting ready
          2. How to do it
          3. See also
        11. Managing images with a private registry
          1. Getting ready
          2. How to do it…
            1. Pushing images
            2. Pulling images
          3. See also
      7. 7. Using Jenkins for Continuous Deployment
        1. Introduction
        2. Installing Jenkins
          1. Getting ready
          2. How to do it…
          3. See also…
        3. Installing the Git plugin
          1. Getting ready
          2. How to do it…
          3. See also
        4. Installing a Jenkins slave
          1. Getting ready
          2. How to do it…
          3. See also
        5. Creating your first Jenkins job
          1. Getting ready
          2. How to do it…
          3. See also
        6. Building Docker containers using Jenkins
          1. Getting ready
          2. How to do it…
        7. Deploying a Java application to Tomcat with zero downtime using Ansible
          1. Getting ready
          2. How to do it…
          3. See also
      8. 8. Metric Collection with InfluxDB
        1. Introduction
        2. Installing InfluxDB
          1. Getting ready
          2. How to do it…
          3. See also
        3. Creating a new InfluxDB database
          1. Getting ready
          2. How to do it…
          3. See also
        4. Logging events with the InfluxDB REST API
          1. Getting ready
          2. How to do it…
          3. See also
        5. Gathering host statistics with Telegraf
          1. Getting ready
          2. How to do it…
          3. See also
        6. Exploring data with the InfluxDB data explorer
          1. Getting ready
          2. How to do it…
          3. See also
        7. Installing Grafana
          1. Getting ready…
          2. How to do it…
          3. See also
        8. Creating dashboards with Grafana
          1. Getting ready
          2. How to do it…
          3. See also
      9. 9. Log Management
        1. Introduction
        2. Centralizing logs with Syslog
          1. Getting ready
          2. How to do it…
          3. See also
        3. Using syslog templates
          1. Getting ready
          2. How to do it…
          3. See also
        4. Managing log rotation with the Logrotate utility
          1. Getting ready
          2. How to do it…
          3. See also
        5. Installing ElasticSearch, Logstash, and Kibana
          1. Getting ready
          2. How to do it…
          3. See also
        6. Importing logs into Elasticsearch with Logstash
          1. Getting ready
          2. How to do it…
          3. See also
        7. Using Kibana queries to explore data
          1. Getting ready
          2. How to do it…
          3. See also
        8. Using Kibana queries to examine data
          1. Getting ready
          2. How to do it…
          3. See also
      10. 10. Monitoring with Sensu
        1. Introduction
        2. Installing a Sensu server
          1. Getting ready
          2. How to do it…
          3. See also
        3. Installing a Sensu client
          1. Getting ready
          2. How to do it…
          3. See also
        4. Installing check prerequisites
          1. Getting ready
          2. How to do it…
        5. Finding community checks
          1. Getting ready
          2. How to do it…
          3. See also
        6. Adding a DNS check
          1. Getting ready
          2. How to do it…
          3. See also
        7. Adding a disk check
          1. Getting ready
          2. How to do it…
          3. See also
        8. Adding a RAM check
          1. Getting ready
          2. How to do it…
          3. See also
        9. Adding a process check
          1. Getting ready…
          2. How to do it…
          3. See also
        10. Adding a CPU check
          1. Getting ready
          2. How to do it…
          3. See also
        11. Creating e-mail alerts
          1. Getting ready
          2. How to do it…
          3. See also
        12. Creating SMS alerts
          1. Getting ready
          2. How to do it…
          3. See also
        13. Using Ansible to install Sensu
          1. Getting ready
          2. How to do it…
          3. See also
      11. 11. IAAS with Amazon AWS
        1. Introduction
        2. Signing up for AWS
          1. Getting ready
          2. How to do it…
          3. See also
        3. Setting up IAM
          1. Getting ready
          2. How to do it…
          3. See also
        4. Creating your first security group
          1. Getting ready
          2. How to do it…
          3. See also
        5. Creating your first EC2 host
          1. Getting ready
          2. How to do it…
          3. See also
        6. Using Elastic Load Balancers
          1. Getting ready
          2. How to do it…
          3. See also
        7. Managing DNS with route53
          1. Getting ready…
          2. How to do it…
          3. See also
        8. Using Ansible to create EC2 hosts
          1. Getting ready
          2. How to do it…
          3. See also
      12. 12. Application Performance Monitoring with New Relic
        1. Introduction
        2. Signing up for a New Relic account
          1. Getting ready
          2. How to do it…
          3. See also
        3. Installing the New Relic Java agent
          1. Getting ready
          2. How to do it…
          3. See also
        4. Using the performance overview
          1. Getting ready
          2. How to do it…
          3. See also
        5. Locating performance bottlenecks with Transaction Traces
          1. Getting ready
          2. How to do it…
          3. See also
        6. Observing database performance with New Relic
          1. Getting ready
          2. How to do it…
          3. See also
        7. Release performance monitoring with New Relic
          1. Getting ready
          2. How to do it…
          3. See also
        8. Server Monitoring with New Relic
          1. Getting ready
          2. How to do it…
          3. See also
    8. III. Module 3
      1. 1. Evolution of a Software House
        1. A brief history of ACME systems
        2. ACME systems version 1.0
          1. Software delivery process flow version 1.0
        3. ACME systems version 2.0
          1. Software delivery process flow version 2.0
          2. A few brave men and women
        4. ACME systems version 3.0
          1. Software delivery process flow version 3.0
        5. ACME systems version 4.0
        6. The evolution in a nutshell
        7. Summary
      2. 2. No Pain, No Gain
        1. Elephant in the room
        2. Defining the rules
        3. Including (almost) everyone
          1. Identifying the key people
          2. Too many cooks
        4. Openness, transparency, and honesty
          1. Location, location, location
          2. It's all happy-clappy management waffle – isn't it?
        5. The great elephant disclosure
          1. Value stream mapping
        6. Summary
      3. 3. Plan of Attack
        1. Setting and communicating the goal and vision
        2. Standardizing vocabulary and language
        3. A business change project in its own right
        4. The merits of a dedicated team
          1. Who to include
        5. The importance of evangelism
        6. Courage and determination
        7. Understanding the cost
        8. Seeking advice from others
        9. Summary
      4. 4. Culture and Behaviors
        1. All roads lead to culture
        2. An open, honest, and safe environment
          1. Openness and honesty
          2. Courageous dialogue
          3. The physical environment
        3. Encouraging and embracing collaboration
        4. Fostering innovation and accountability at grass roots
        5. The blame culture
          1. Blame slow, learn quickly
        6. Building trust-based relationships across organizational boundaries
        7. Rewarding good behaviors and success
          1. The odd few
        8. Recognizing dev and ops teams are incentivized can have an impact
        9. Embracing change and reducing risk
          1. Changing people's perceptions with pudding
        10. Being transparent
        11. Summary
      5. 5. Approaches, Tools, and Techniques
        1. Engineering best practice
          1. Source control
          2. Small, frequent, and simple changes
          3. Never break your consumer
          4. Open and honest peer-working practices
          5. Fail fast and often
          6. Automated builds and tests
          7. Continuous Integration
          8. Using the same binary across all environments
        2. How many environments are enough?
        3. Developing against a production-like environment
        4. CD tooling
          1. Automated provisioning
          2. No-downtime deployments
        5. The cloud
        6. Monitoring
        7. When a simple manual process is also an effective tool
        8. Summary
      6. 6. Hurdles Along the Way
        1. What are the potential issues you need to look out for?
          1. Dissenters in the ranks
            1. No news is no news
            2. The anti-agile brigade
          2. The transition curve
          3. The outsiders
          4. Corporate guidelines, red tape, and standards
          5. Geographically diverse teams
          6. Failure during evolution
          7. Processes that are not repeatable
          8. Recruitment
        2. Summary
      7. 7. Vital Measurements
        1. Measuring effective engineering best practice
          1. Simple quality metrics
          2. Code complexity
          3. Unit test coverage
          4. Commit rates
          5. Adherence to coding rules and standards
          6. Where to start and why bother?
        2. Measuring the real world
          1. Measuring the stability of the environments
            1. Incorporating automated tests
            2. Combining automated tests and system monitoring
            3. Real-time monitoring of the software itself
            4. Monitoring utopia
          2. Effectiveness of CD and DevOps
          3. Impact of CD and DevOps
          4. Measuring your culture
        3. Summary
      8. 8. Are We There Yet?
        1. Reflect on where you are now
        2. Streaming
        3. A victim of your own success
        4. [P]lan, [D]o, [C]heck, [A]djust
        5. Exit stage left
        6. Rest on your laurels (not)
        7. Summary
      9. 9. The Future is Bright
        1. Expanding your horizon
          1. Reactive performance and load testing
          2. Reducing feature flag complexity
          3. Easing A/B testing
          4. Security patching and saving your bacon
          5. Order out of chaos monkey
          6. End user self-service
          7. CD and DevOps and the mobile world
        2. Expanding beyond software delivery
        3. What about me?
        4. What have you learned?
        5. Summary
      10. A. Some Useful Information
        1. Tools
        2. People
        3. Recommended reading
      11. B. Where Am I on the Evolutionary Scale?
      12. C. Retrospective Games
        1. The timeline game
        2. StoStaKee
      13. D. Vital Measurements Expanded
        1. Code complexity – some science
        2. Code versus comments
        3. Embedding monitoring into your software
    9. A. Bibliography
    10. Index