Learning Puppet 4

Learning Puppet 4

by Jo Rhett
Released April 2016
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781491907665

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

If you're a system administrator, developer, or site reliability engineer responsible for handling hundreds or even thousands of nodes in your network, the Puppet configuration management tool will make your job a whole lot easier. This practical guide shows you what Puppet does, how it works, and how it can provide significant value to your organization. Through hands-on tutorials, DevOps engineer Jo Rhett demonstrates how Puppet manages complex and distributed components to ensure service availability.

Publisher resources

View/Submit Errata

Table of contents

  1. Foreword
  2. Preface
    1. Who This Book Is For
    2. What to Expect from Me
    3. What You Will Need
    4. What You’ll Find in This Book
    5. How to Use This Book
    6. IPv6 Ready
    7. SSL is now TLS
    8. Conventions Used in This Book
    9. Safari® Books Online
    10. How to Contact Us
    11. Content Updates
      1. February 9, 2018
    12. Acknowledgments
  3. Introduction
    1. What Is Puppet?
    2. Why Declarative
    3. How Puppet Works
    4. Why Use Puppet
    5. Is Puppet DevOps?
    6. Time to Get Started
  4. I. Controlling with Puppet Apply
  5. 1. Thinking Declarative
    1. Handling Change
    2. Using Idempotence
    3. Declaring Final State
    4. Reviewing Declarative Programming
  6. 2. Creating a Learning Environment
    1. Installing Vagrant
      1. Installing Vagrant on Mac
      2. Installing Git Tools on Windows
      3. Installing VirtualBox on Windows
      4. Installing Vagrant on Windows
    2. Starting a Bash Shell
    3. Downloading a Box
    4. Cloning the Learning Repository
    5. Install the Vagrant vbguest Plugin
    6. Initializing the Vagrant Setup
    7. Verifying the /vagrant Filesystem
    8. Initializing Non-Vagrant Systems
    9. Installing Some Helpful Utilities
    10. Choosing a Text Editor
      1. On the Virtual System
      2. On Your Desktop
      3. In Your Profile
    11. Reviewing the Learning Environment
  7. 3. Installing Puppet
    1. Adding the Package Repository
      1. What Is a Puppet Collection?
    2. Installing the Puppet Agent
    3. Reviewing Dependencies
    4. Reviewing Puppet 4 Changes
      1. Linux and Unix
      2. Windows
    5. Making Tests Convenient
    6. Running Puppet Without sudo
    7. Running Puppet with sudo
    8. Reviewing Puppet Installation
  8. 4. Writing Manifests
    1. Implementing Resources
    2. Applying a Manifest
    3. Declaring Resources
    4. Viewing Resources
    5. Executing Programs
      1. Was That Idempotent?
    6. Managing Files
      1. Finding File Backups
      2. Restoring Files
    7. Avoiding Imperative Manifests
    8. Testing Yourself
    9. Reviewing Writing Manifests
  9. 5. Using the Puppet Configuration Language
    1. Defining Variables
      1. Defining Numbers
      2. Creating Arrays and Hashes
      3. Mapping Hash Keys and Values
      4. Using Variables in Strings
      5. Redacting Sensitive Values
      6. Using Braces to Limit Problems
      7. Preventing Interpolation
      8. Using Unicode Characters
      9. Avoiding Redefinition
      10. Avoiding Reserved Words
      11. Learning More
    2. Finding Facts
    3. Calling Functions in Manifests
    4. Using Variables in Resources
    5. Defining Attributes with a Hash
    6. Declaring Multiple Resource Titles
    7. Declaring Multiple Resource Bodies
    8. Modifying with Operators
      1. Adding to Arrays and Hashes
      2. Removing from Arrays and Hashes
      3. Order of Operations
    9. Using Comparison Operators
    10. Evaluating Conditional Expressions
    11. Matching Regular Expressions
    12. Building Lambda Blocks
    13. Looping Through Iterations
      1. each()
      2. filter()
      3. map()
      4. reduce()
      5. slice()
      6. with()
      7. Capturing Extra Parameters
      8. Terminating Iteration Evaluation
      9. Iteration Wrap-Up
    14. Reviewing Puppet Configuration Language
  10. 6. Controlling Resource Processing
    1. Adding Aliases
      1. Specifying an Alias by Title
      2. Adding an Alias Metaparameter
    2. Preventing Action
    3. Auditing Changes
    4. Defining Log Level
    5. Filtering with Tags
      1. Skipping Tags
    6. Limiting to a Schedule
      1. Utilizing periodmatch
      2. Avoiding Dependency Failures
    7. Declaring Resource Defaults
    8. Reviewing Resource Processing
  11. 7. Expressing Relationships
    1. Managing Dependencies
    2. Referring to Resources
    3. Ordering Resources
    4. Assuming Implicit Dependencies
    5. Triggering Refresh Events
    6. Chaining Resources with Arrows
    7. Processing with Collectors
    8. Understanding Puppet Ordering
    9. Debugging Dependency Cycles
      1. Avoiding the Root User Trap
      2. Utilizing Stages
    10. Reviewing Resource Relationships
  12. 8. Upgrading from Puppet 3
    1. Replacing Deprecated Features
      1. Junking the Ruby DSL
      2. Upgrading Config Environments
      3. Removing Node Inheritence
      4. Disabling puppet kick
      5. Qualifying Relative Class Names
      6. Losing the Search Function
      7. Replacing Import
      8. Documenting Modules with Puppet Strings
      9. Installing the Tagmail Report Processor
      10. Querying PuppetDB
    2. Preparing for the Upgrade
      1. Validating Variable Names
      2. Quoting Strings
      3. Preventing Numeric Assignment
      4. Testing Boolean Facts
      5. Qualifying Defined Types
      6. Adding Declarative Permissions
      7. Removing Cron Purge
      8. Replacing MSI Package Provider
      9. Adjusting Networking Facts
    3. Testing with the Future Parser
      1. Using Directory Environments
      2. Duplicating a Master or Node
    4. Enhancing Older Manifests
      1. Adding else to unless
      2. Calling Functions in Strings
      3. Matching String Regexps
      4. Letting Expressions Stand Alone
      5. Chaining Assignments
      6. Chaining Expressions with a Semicolon
      7. Using Hash and Array Literals
      8. Configuring Error Reporting
  13. 9. Wrap-Up of Puppet Basics
    1. Best Practices for Writing Manifests
    2. Learning More About Puppet Manifests
  14. II. Creating Puppet Modules
  15. 10. Creating a Test Environment
    1. Verifying the Production Environment
    2. Creating the Test Environment
    3. Changing the Base Module Path
    4. Skipping Ahead
  16. 11. Separating Data from Code
    1. Introducing Hiera
    2. Selecting a Data Source
      1. Selecting a Hiera Backend
      2. Creating Hiera Data
    3. Configuring Hiera
      1. Version
      2. Defaults
      3. Hierarchy
      4. Merge Strategy
      5. Complete Example
    4. Looking Up Hiera Data
      1. Checking Hiera Values from the Command Line
      2. Performing Hiera Lookups in a Manifest
      3. Testing Merge Strategy
    5. Providing Global Data
  17. 12. Using Modules
    1. Finding Modules
      1. Puppet Forge
      2. Public GitHub Repositories
      3. Internal Repositories
    2. Evaluating Module Quality
      1. Puppet Supported
      2. Puppet Approved
      3. Quality Score
      4. Community Rating
    3. Installing Modules
      1. Installing from a Puppet Forge
      2. Installing from GitHub
    4. Testing a Single Module
    5. Defining Config with Hiera
    6. Assigning Modules to Nodes
      1. Using Hiera for Module Assignment
      2. Assigning Classes to Every Node
      3. Altering the Class List per Node
      4. Avoiding Node Assignments in Manifests
      5. Upgrading from Puppet 2 or 3
    7. Examining a Module
    8. Reviewing Modules
  18. 13. Designing a Custom Module
    1. Choosing a Module Name
      1. Avoiding Reserved Names
    2. Generating a Module Skeleton
      1. Modifying the Default Skeleton
    3. Understanding Module Structure
    4. Installing the Module
    5. Creating a Class Manifest
      1. What Is a Class?
    6. Declaring Class Resources
    7. Accepting Input
    8. Sharing Files
    9. Testing File Synchronization
    10. Synchronizing Directories
    11. Parsing Templates
      1. Common Syntax
      2. Using Puppet EPP Templates
      3. Using Ruby ERB Templates
      4. Creating Readable Templates
    12. Testing the Module
    13. Peeking Beneath the Hood
    14. Best Practices for Module Design
    15. Reviewing Custom Modules
  19. 14. Improving the Module
    1. Validating Input with Data Types
      1. Valid Types
      2. Validating Values
      3. Testing Values
      4. Comparing Strings with Regular Expressions
      5. Matching a Regular Expression
      6. Revising the Module
    2. Looking Up Input from Hiera
      1. Naming Parameters Keys Correctly
      2. Using Array and Hash Merges
      3. Understanding Lookup Merge
      4. Debugging Lookup
      5. Specifying Merge Strategy in Data
      6. Replacing Direct Hiera Calls
    3. Building Subclasses
    4. Creating New Resource Types
    5. Understanding Variable Scope
      1. Using Out-of-Scope Variables
      2. Understanding Top Scope
      3. Understanding Node Scope
      4. Understanding Parent Scope
      5. Tracking Resource Defaults Scope
      6. Avoiding Resource Default Bleed
      7. Redefining Variables
    6. Calling Other Modules
      1. Sourcing a Common Dependency
      2. Using a Different Module
    7. Ordering Dependencies
      1. Depending on Entire Classes
      2. Placing Dependencies Within Optional Classes
      3. Notifying Dependencies from Dynamic Resources
      4. Solving Unknown Resource Dependencies
    8. Containing Classes
    9. Creating Reusable Modules
      1. Avoiding Fixed Values in Attribute Values
      2. Ensuring Fixed Values for Resource Names
      3. Defining Defaults in a Params Manifest
    10. Best Practices for Module Improvements
    11. Reviewing Module Improvements
  20. 15. Extending Modules with Plugins
    1. Adding Custom Facts
      1. External Facts
      2. Custom (Ruby) Facts
      3. Debugging
      4. Understanding Implementation Issues
    2. Defining Functions
      1. Puppet Functions
      2. Ruby Functions
      3. Using Custom Functions
    3. Creating Puppet Types
      1. Defining Ensurable
      2. Accepting Params and Properties
      3. Validating Input Values
      4. Defining Implicit Dependencies
      5. Learning More About Puppet Types
    4. Adding New Providers
      1. Determining Provider Suitability
      2. Assigning a Default Provider
      3. Defining Commands for Use
      4. Ensure the Resource State
      5. Adjusting Properties
      6. Providing a List of Instances
      7. Taking Advantage of Caching
      8. Learning More About Puppet Providers
    5. Identifying New Features
    6. Binding Data Providers in Modules
      1. Using Data from Hiera
      2. Performing Lookup Queries
    7. Requirements for Module Plugins
    8. Reviewing Module Plugins
  21. 16. Documenting Modules
    1. Learning Markdown
    2. Writing a Good README
    3. Documenting the Classes and Types
      1. Installing YARD and Puppet Strings
      2. Fixing the Headers
      3. Listing Parameters
      4. Documenting Variable References
      5. Showing Examples
      6. Listing Authors and Copyright
    4. Documenting Functions
    5. Generating Documentation
    6. Updating Module Metadata
      1. Identifying the License
      2. Promoting the Project
      3. Indicating Compatibility
      4. Defining Requirements
      5. Listing Dependencies
      6. Identifying a Module Data Source
      7. Updating Old Metadata
    7. Maintaining the Change Log
    8. Evolving and Improving
    9. Best Practices for Documenting Modules
  22. 17. Testing Modules
    1. Installing Dependencies
      1. Installing Ruby
      2. Adding Beaker
      3. Bundling Dependencies
    2. Preparing Your Module
      1. Defining Fixtures
    3. Defining RSpec Unit Tests
      1. Defining the Main Class
      2. Passing Valid Parameters
      3. Failing Invalid Parameters
      4. Testing File Creation
      5. Validating Class Inclusion
      6. Using Facts in Tests
      7. Using Hiera Input
      8. Defining Parent Class Parameters
      9. Testing Functions
      10. Adding an Agent Class
      11. Testing Other Types
    4. Creating Acceptance Tests
      1. Installing Ruby for System Tests
      2. Defining the Nodeset
      3. Configuring the Test Environment
      4. Creating an Acceptance Test
      5. Running Acceptance Tests
    5. Using Skeletons with Testing Features
    6. Finding Documentation
    7. Reviewing Testing Modules
  23. 18. Publishing Modules
    1. Updating the Module Metadata
    2. Packaging a Module
    3. Uploading a Module to the Puppet Forge
    4. Publishing a Module on GitHub
    5. Automating Module Publishing
    6. Getting Approved Status from Puppet Labs
  24. III. Using a Puppet Server
  25. 19. Preparing for a Puppet Server
    1. Understanding the Catalog Builder
      1. Node
      2. Agent
      3. Server
    2. Planning for Puppet Server
      1. The Server Is Not the Node
      2. The Node Is Not the Server
      3. Store Server Data Files Separately
      4. Functions Run on the Server
    3. Choosing Puppet Master Versus Puppet Server
      1. Upgrading Easily with Puppet Master
      2. Embracing the Future with Puppet Server
      3. Why There’s Really No Choice
    4. Ensuring a High-Performance Server
  26. 20. Creating a Puppet Master
    1. Starting the puppetmaster VM
    2. Installing the Puppet Master
    3. Configuring a Firewall for the Puppet Master
    4. Running the WEBrick Server
    5. Testing with the Puppet Master Service
    6. Scaling the Puppet Master with Passenger
      1. Installing Apache
      2. Installing Phusion Passenger
      3. Configuring the Puppet Master
    7. IPv6 Dual-Stack Puppet Master
    8. Debugging Puppet Master
  27. 21. Creating a Puppet Server
    1. Starting the puppetserver VM
    2. Installing Puppet Server
    3. Configuring a Firewall for Puppet Server
    4. Configuring Puppet Server
      1. Defining Server Paths
      2. Limiting Memory Usage
      3. Configuring TLS Certificates
      4. Avoiding Obsolete Settings
      5. Configuring Server Logs
      6. Configuring Server Authentication
    5. Running Puppet Server
      1. Adding Ruby Gems
    6. IPv6 Dual-Stack Puppet Server
  28. 22. Connecting a Node
    1. Creating a Key Pair
    2. Authorizing the Node
    3. Downloading the First Catalog
    4. Installing Hiera Data and Modules
    5. Testing with a Client Node
    6. Learning More About Puppet Server
  29. 23. Migrating an Existing Puppet Master
    1. Migrating the Puppet Master Config
    2. Synchronizing All Environments
    3. Copying Hiera Data
    4. Moving the MCollective Config Directory
    5. Removing Node Inheritance
    6. Testing a Client Node
    7. Upgrading Clients
  30. 24. Utilizing Advantages of a Puppet Server
    1. Using Server Data in Your Manifests
      1. Trusted Facts
      2. Server Facts
      3. Server Configuration Settings
    2. Backing Up Files Changed on Nodes
    3. Processing Puppet Node Reports
      1. Enabling Transmission of Reports
      2. Running Audit Inspections
      3. Storing Node Reports
      4. Logging Node Reports
      5. Transmitting Node Reports via HTTP
      6. Transmitting Node Reports to PuppetDB
      7. Emailing Node Reports
      8. Creating a Custom Report Processor
  31. 25. Managing TLS Certificates
    1. Reviewing Node Authentication
    2. Autosigning Agent Certificates
      1. Name-Based Autosigning
      2. Policy-Based Autosigning
      3. Naive Autosigning
    3. Using an External Certificate Authority
      1. Distributing Certificates Manually
      2. Installing Certificates on the Server
      3. Disabling CA on a Puppet Server
      4. Disabling CA on a Puppet Master
      5. Using Different CAs for Servers and Agents
      6. Distributing the CA Revocation List
    4. Learning More About TLS Authentication
  32. 26. Growing Your Puppet Deployment
    1. Using a Node Terminus
      1. Running an External Node Classifier
      2. Querying LDAP
      3. Starting with Community Examples
    2. Deploying Puppet Servers at Scale
      1. Keeping Distinct Domains
      2. Sharing a Single Puppet CA
      3. Using a Load Balancer
      4. Managing Geographically Dispersed Servers
      5. Managing Geographically Dispersed Nodes
      6. Falling Back to Cached Catalogs
      7. Making the Right Choice
    3. Best Practices for Puppet Servers
    4. Reviewing Puppet Servers
  33. IV. Integrating Puppet
  34. 27. Tracking Puppet Status with Dashboards
    1. Using Puppet Dashboard
      1. Installing Dashboard Dependencies
      2. Enabling Puppet Dashboard
      3. Viewing node status
      4. Using Dashboard as a Node Classifier
      5. Implementing Dashboard in Production
    2. Evaluating Alternative Dashboards
      1. Puppetboard
      2. Puppet Explorer
      3. PanoPuppet
      4. ENC Dashboard
      5. Foreman
    3. Upgrading to the Enterprise Console
      1. Viewing Status
      2. Classifying Nodes
      3. Inspecting Events
      4. Tracking Changes
      5. Controlling Access
      6. Evaluating Puppet Enterprise
    4. Finding Plugins and Tools
  35. 28. Running the Puppet Agent on Windows
    1. Creating a Windows Virtual Machine
      1. Creating a VirtualBox Windows VM
      2. Adding an Internal Network Adapter
      3. Connecting the Windows Installation Media
      4. Configuring the Internal Network Adapter
    2. Installing Puppet on Windows
    3. Configuring Puppet on Windows
    4. Running Puppet Interactively
    5. Starting the Puppet Service
    6. Debugging Puppet Problems
    7. Writing Manifests for Windows
    8. Finding Windows-Specific Modules
    9. Concluding Thoughts on Puppet Windows
  36. 29. Customizing Environments
    1. Understanding Environment Isolation
    2. Enabling Directory Environments
    3. Assigning Environments to Nodes
    4. Configuring an Environment
      1. Environment Configuration File
      2. Choosing a Manifest Path
      3. Disabling the Environment Cache
    5. Using Environment Data
      1. Removing Environment Paths from Global Data
      2. Utilizing Hiera Hierarchy in Environments
      3. Using Custom Backends in Environments
      4. Querying Environment Data from Hiera
    6. Strategizing How to Use Environments
      1. Promoting Change Through Layers
      2. Solving One-Off Problems Using Environments
      3. Supporting Diverse Teams with Environments
    7. Managing Environments with r10k
      1. Listing Modules in the Puppetfile
      2. Creating a Control Repository
      3. Configuring r10k Sources
      4. Adding New Environments
      5. Populating a New Installation
      6. Updating a Single Environment
      7. Replicating Hiera Data
    8. Invalidating the Environment Cache
    9. Restarting JRuby When Updating Plugins
    10. Reviewing Environments
  37. 30. Controlling Puppet with MCollective
    1. Configuring MCollective
      1. Enabling the Puppet Labs Repository
      2. Installing the MCollective Module
      3. Generating Passwords
      4. Configuring Hiera for MCollective
      5. Enabling the Middleware
      6. Connecting MCollective Servers
      7. Validating the Installation
      8. Creating Another Client
      9. Installing MCollective Agents and Clients
      10. Sharing Facts with Puppet
    2. Pulling the Puppet Strings
      1. Viewing Node Inventory
      2. Checking Puppet Status
      3. Disabling the Puppet Agent
      4. Invoking Ad Hoc Puppet Runs
      5. Limiting Targets with Filters
      6. Providing a List of Targets
      7. Limiting Concurrency
      8. Manipulating Puppet Resource Types
    3. Comparing to Puppet Application Orchestration
    4. Learning More About MCollective
  38. 31. Managing Network Infrastructure with Puppet
    1. Managing Network Devices with Puppet Device
      1. Enabling SSH on the Switch
      2. Configuring the Puppet Proxy Agent
      3. Installing the Device_Hiera Module
      4. Defining Resource Defaults in Hiera
      5. Centralizing VLAN Configuration
      6. Applying Default Configs to Interfaces
      7. Customizing Interface Configurations
      8. Testing Out the Switch Configuration
      9. Adding Resource Types and Providers
      10. Merging Defaults with Other Resources
    2. Using the NetDev Standard Library
      1. Finding NetDev Vendor Extensions
      2. Creating a NetDev Device Object
      3. Reducing Duplication with Device_Hiera
    3. Puppetizing Cisco Nexus Switches
      1. Configuring the Puppet Server
      2. Preparing the NX-OS Device
      3. Installing the NX-OS Puppet Agent
      4. Enabling the NX-OS Puppet Agent
      5. Managing Configuration
    4. Puppetizing Juniper Devices
      1. Supported Devices
      2. Installing Modules on the Puppet Server
      3. Preparing the Junos Device
      4. Installing the Junos Puppet Agent
      5. Creating the Puppet User
      6. Adjusting Physical Interface Settings
      7. Simplifying Layer-2 VLANs
      8. Enabling Link Aggregation
      9. Defining Ad Hoc Configuration Parameters
      10. Distributing Junos Event Scripts
      11. Running Puppet Automatically
      12. Troubleshooting
    5. Best Practices for Network Devices
    6. Reviewing Network Devices
  39. 32. Assimilating Puppet Best Practices
    1. Managing Change
      1. Expecting Change
      2. Controlling Rate of Change
      3. Tracking Change
    2. Choosing Puppet Apply Versus Puppet Server
      1. Benefits of Puppet Apply
      2. Benefits of Puppet Server
      3. Benefits Shared
      4. Summarizing the Differences
    3. Creating a Private Puppet Forge
      1. Pulp
      2. Puppet Forge Server
      3. Django Forge
    4. Good Practices
      1. Indenting Heredoc
      2. Splaying Puppet Agent Cron Jobs
      3. Cleaning Puppet Reports
      4. Trimming the File Bucket
    5. Drinking the Magic Monkey Juice
      1. Hating on Params.pp
      2. Disabling Environments
      3. Tracking Providers
      4. Breaking the Rules
      5. Working Good, Fast, Cheap
      6. Choosing Fight or Flight
      7. Letting the Strings Pull You
      8. Leveraging Puppet for Small Changes
      9. Tossing Declarative to the Wind
      10. Allowing Anyone to sudo puppet
  40. 33. Finding Support Resources
    1. Accessing Community Support
    2. Engaging Puppet Labs Support
    3. Contacting the Author
  41. Afterword
    1. Some Best Practices May Not Work for You
    2. Learning to Fail is the Secret to Success
  42. A. Installing Puppet on Other Platforms
    1. Debian and Ubuntu
    2. Fedora
    3. Other Platforms
  43. B. Configuring Firewalls on Other Platforms
    1. IP Tables
    2. Uncomplicated Firewall
  44. C. Installing Ruby
    1. Ruby for Mac
    2. Ruby for Windows
    3. Ruby for Linux
  45. Index

Product information

  • Title: Learning Puppet 4
  • Author(s): Jo Rhett
  • Release date: April 2016
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781491907665