O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Python for DevOps

Book Description

Much has changed in technology over the past decade. Data is hot, the cloud is ubiquitous, and many organizations need some form of automation. Throughout all these transformations, Python has become one of the most popular languages in the world. This practical guide shows you how to use Python for everyday Linux systems administration tasks with today’s most useful DevOps tools, including Docker, Kubernetes, and Terraform.

Learning how to interact and automate with Linux is an essential skill for millions of professionals. Python makes it much easier. With this book, you’ll learn how to develop software and solve problems using containers, as well as how to monitor, instrument, load-test, and operationalize your software. If you’re looking for effective ways to "get stuff done" in Python, this is your guide.

  • Automate several tasks using Python
  • Work more efficiently by using a smaller subset of the language
  • Use continuous integration systems to increase software quality
  • Mix shell and Python commands to solve problems

Table of Contents

  1. 1. Introduction
    1. Why Read This Book?
    2. What does DevOps mean to the authors?
    3. How to use this book?
    4. An Overview of Python Essentials
    5. Installing and Running Python
      1. The Python Shell
      2. Jupyter Notebooks
    6. Procedural Programming
      1. Variables
      2. Basic Math
      3. Comments
    7. Built-in Functions
      1. Print
      2. Range
    8. Execution Control
      1. if/elif/else
      2. For Loops
    9. While Loops
    10. Handling Exceptions
    11. Built in Objects
      1. What is an object
      2. Object Methods and Attributes
      3. Sequences
    12. Functions
      1. Anatomy of a Function
      2. Function as Objects
      3. Wrapping Functions with Decorators
      4. Anonymous Functions
    13. RE Package
      1. Searching
      2. Character sets
      3. Character Classes
      4. Groups
      5. Named Groups
      6. Find All
      7. Find Iterator
      8. Substitution
      9. Compiling
    14. Lazy Evaluation
      1. generators
      2. generator comprehensions
    15. More IPython Features
      1. Using IPython To Run Unix Shell Commands
    16. Summary
  2. 2. Automating Text and Files
    1. Reading and Writing Files
      1. Using Regular Expressions to search Text
      2. Dealing with large files
      3. Encrypting Text
      4. Hashing with Hashlib
      5. Encryption with Cryptography
      6. Summary
  3. 3. Working With The Command Line
    1. Working with the Shell
      1. Talking to the Interpreter with the Sys Module
      2. Dealing with the Operating System using the OS Module
      3. Spawn Processes with the Subprocess Module
    2. Creating Commandline Tools
      1. Using sys.argv
      2. Using argparse
      3. Using click
      4. Fire
      5. Implementing plugins
    3. Case Study: Turbocharging Python with Command Line Tools
      1. Using The Numba JIT (Just in time Compiler)
      2. Using the GPU with CUDA Python
      3. Running True Multi-Core Multithreaded Python using Numba
      4. KMeans Clustering
    4. Summary
  4. 4. Package Management
    1. Why is packaging important?
    2. Packaging guidelines
      1. Descriptive Versioning
      2. The changelog
    3. Choosing a strategy
    4. Packaging solutions
      1. Native Python packaging
      2. Debian packaging
      3. RPM packaging
    5. Management with systemd
      1. Long running processes
      2. Setting it up
      3. The systemd unit file
      4. Unit
      5. Service
      6. Install
    6. Installing the unit
      1. Log handling
    7. References
  5. 5. Continuous Integration and Continuous Deployment
    1. Real-World Case Study: Converting A Poorly Maintained Word Press Site to Hugo
      1. Setup Hugo
      2. Convert Wordpress to Hugo Posts
      3. Create an Algolia Index and Update It
      4. Orchestrating with a Makefile
      5. Deploying with AWS Code Pipeline
    2. Real-World Case Study: Deploying a Python App Engine Application with Google Cloud Build
    3. Real-World Case Study: NFSOPS
  6. 6. Pytest for DevOps
    1. Testing superpowers with pytest
    2. Getting started with pytest
      1. Testing with pytest
      2. Differences with unittest
    3. pytest features
      1. conftest.py
      2. The amazing assert
      3. Parametrize
    4. Fixtures
      1. Getting started
      2. Built-in fixtures
    5. Infrastructure testing
      1. What is system validation?
      2. Introduction to testinfra
      3. Connecting to remote nodes
      4. Features and special Fixtures
    6. Examples
    7. Testing Jupyter Notebooks w/ PyTest
    8. References
  7. 7. Cloud Computing
    1. Cloud Computing Foundations
      1. Virtualization & Containers
      2. Challenges and Opportunities in Distributed Computing
    2. Conclusion
    3. Exercises
    4. Case Study Questions
  8. 8. Infrastructure As Code
    1. A Classification of Infrastructure Automation Tools
    2. Manual Provisioning
    3. Automated Infrastructure Provisioning with Terraform
      1. Provisioning an S3 Bucket
      2. Provisioning an SSL Certificate with AWS ACM
      3. Provisioning an AWS CloudFront Distribution
      4. Provisioning a Route53 DNS Record
      5. Copying Static Files to S3
      6. Deleting All AWS Resources Provisioned with Terraform
    4. Automated Infrastructure Provisioning with Pulumi
      1. Creating a New Pulumi Python Project for AWS
      2. Creating Configuration Values for the Staging Stack
      3. Provisioning an ACM SSL Certificate
      4. Provisioning a Route53 Zone and DNS Records
      5. Provisioning a CloudFront Distribution
      6. Provisioning a Route53 DNS Record for the Site URL
      7. Creating and Deploying a New Stack
    5. Summary
  9. 9. Container Technologies: Docker And Docker-Compose
    1. What Is A Docker Container
    2. Creating, Building, Running And Removing Docker Images And Containers
    3. Publishing Docker Images To A Docker Registry
    4. Running a Docker Container with the Same Image on a Different Host
    5. Running Multiple Docker Containers with docker-compose
    6. Porting the docker-compose Services to a New Host and Operating System
  10. 10. Container Orchestration: Kubernetes
    1. Short Overview of Kubernetes Concepts
    2. Using Kompose to Create Kubernetes Manifests from docker-compose.yaml
    3. Deploying Kubernetes Manifests to a Local Kubernetes Cluster Based on minikube
    4. Launching a GKE Kubernetes Cluster in GCP with Pulumi
    5. Deploying the Flask Example Application to GKE
    6. Installing Prometheus and Grafana Helm Charts
    7. Destroy the GKE Cluster
    8. Summary
    9. Exercises
  11. 11. Serverless Technologies
    1. Learning Objectives for this Chapter
    2. Deploying the Same Python Function to the “Big Three” Cloud Providers
      1. Install Serverless Framework
      2. Deploy Python Function to AWS Lambda
      3. Deploy Python Function to Google Cloud Functions
      4. Deploy Python Function to Azure
    3. Deploying a Python Function to Self-Hosted FaaS platforms
      1. Deploy Python function to Apache OpenWhisk
      2. Deploy Python function to OpenFaaS
    4. Provision DynamoDB Table, Lambda Functions and API Gateway Methods Using the AWS CDK
    5. Exercises