O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Learning MCollective

Learning MCollective

by Jo Rhett
Publisher: O'Reilly Media, Inc.
Release Date: August 2014
ISBN: 9781491945681
View table of contents
Start reading

Book Description

Orchestrate change across server clusters in near realtime with MCollective, the framework that works in concert with Puppet, Chef, and other configuration management tools. Ideal for system administrators and operations or DevOps engineers at any level, this hands-on guide teaches you how to build and test a real installation of MCollective servers and clients in your environment.

Table of Contents

  1. Preface
    1. Who this book is for
    2. What to expect from me
    3. What you’ll find in this book
    4. IPv6 Ready
    5. Let’s Go.
  2. I. Getting Started
    1. 1. Introduction
      1. What is MCollective?
      2. Why Parallel Execution?
      3. How MCollective Works
      4. Why Use MCollective
      5. How to Fail with MCollective
      6. Time to Get Started
    2. 2. Installation
      1. Requirements
        1. MCollective Clients and Servers
        2. Middleware Broker
      2. Passwords and Keys
      3. Puppet Labs Repository
        1. Supported Platforms
          1. Enterprise Linux 6
          2. Enterprise Linux 5
          3. Fedora
          4. Debian and Ubuntu
          5. Other Platforms
      4. Configuring ActiveMQ
        1. Install the Software
        2. Tune the Configuration File
          1. Enable Purging in the Broker
          2. Disable producerFlowControl
          3. Define logins for clients and servers in simpleAuthenticationPlugins
          4. Define permissions for clients and servers in authorizationPlugins
          5. Transports
          6. Disable the Web Console
        3. Start the Service
        4. Firewall Change
      5. Installing Servers
        1. Install the Software
        2. Server Configuration File
        3. Start the Service
      6. Creating a Client
        1. Install the Software
        2. Client Configuration File
        3. Security Considerations
      7. Installing from Source
        1. Using the Installer
        2. Creating an Init Script
        3. Creating a Package
      8. Testing Your Installation
      9. Troubleshooting
        1. Passwords
        2. Networking
    3. 3. Command Line Client
      1. Connector
      2. Inventory
        1. Inventory Reports
      3. Discovery
      4. Filters
        1. Combination Filters
      5. Limits
      6. Output
      7. Facts
      8. Classes
      9. Bash Completion
    4. 4. Agents
      1. Agent Plugins
      2. Connector Plugins
      3. Installing from Packages
      4. Installing from Source
        1. Copy Agents into Location
      5. Notify mcollectived
      6. Disabling Agents
      7. Using Client Plugins
      8. Finding Community Plugins
    5. 5. Maintenance
      1. Keeping Sessions Alive
      2. Activating Changes
      3. Server Statistics
      4. Logging
      5. Monitoring Servers
    6. 6. Configuration Management
      1. Installing the Puppet Module
        1. Using r10k
        2. Straight from GitHub
      2. Configuring MCollective using Puppet
        1. Hiera Configuration Data
      3. Sharing Facts with Puppet
      4. Installing Agents with Puppet
    7. 7. Controlling Puppet
      1. Install the Puppet Agent
      2. Checking Puppet Status
      3. Controlling the Puppet Daemon
      4. Invoking Ad-Hoc Puppet Runs
      5. Manipulating Puppet Resource Types
        1. Restricting which Resources can be Controlled
        2. Block MCollective from Puppet Resources
      6. Debugging
        1. Unable to Match Server with Class
        2. Unable to Match Server with Fact
        3. Unable to Match Server by Hostname
        4. Some servers ignore messages
  3. II. Complex Installations
    1. 8. Middleware Configuration
      1. Open Platform
      2. Network Security
        1. Transport Connectors
        2. Firewall Configurations
      3. IPv6 Dual-Stack Environments
      4. ActiveMQ Config Structure
      5. Detailed Configuration Review
        1. Broker Definition
        2. Topic and Queue Tuning
        3. Authentication and Authorization
          1. Users and Groups
          2. Topics and Queues the Clients Send To
          3. Topics and Queues the Servers Read From
          4. Topics and Queues the Servers Write To
        4. Transport Connectors
        5. Conclusion
      6. ActiveMQ Clusters
        1. Network of Brokers
        2. Master-Slave Redundancy
        3. Encrypted Broker Links
        4. Conclusion
      7. Large-Scale Broker Configurations
        1. Recommendations
        2. Supporting 1000+ Servers
        3. ActiveMQ 5.9.1
        4. Problems Seen in the Wild
        5. Conclusion
    2. 9. Middleware Security
      1. Anonymous TLS Security
        1. Overview
          1. Advantages
          2. Disadvantages
        2. Setup
          1. Puppet One Step Process
          2. Create a Java Keystore by Hand
          3. Configure the middleware to use the keystore
          4. Configure the Client and Server by Hand
        3. Testing
      2. Trusted TLS Servers
        1. Overview
          1. Advantages
          2. Disadvantages
        2. Setup Paths
        3. TLS using Puppet CA
          1. Puppet One Step Process
          2. Create a Java trustStore by Hand
          3. Create a Java Keystore by Hand
          4. Configure the Broker by Hand
          5. Configure the MCollective Server by Hand
        4. TLS using Another CA
          1. Create a new Certificate Authority (optional)
          2. Create a Java Truststore from the Certificate Authority
          3. Create a TLS Keypair for every server
          4. Create a Java Keystore
          5. Configure the broker to use these stores we made
          6. Configure the MCollective Server by Hand
      3. Validate Keystore and Truststore
      4. Trusted TLS Clients
        1. Clients of the Puppet CA
          1. Create a Puppet Keypair on the Client Node
          2. Change the Client Configuration
        2. Clients using another CA
          1. Create a Keypair for each Client
          2. Sign the certificate request
        3. Change the Client Configuration
      5. Conclusion
    3. 10. Creating Collectives
      1. Deciding When to Create More
      2. Collectives != Clustering
      3. Configuration Traffic
      4. Localizing Traffic
      5. Limiting Access
      6. Conclusion
    4. 11. MCollective Security
      1. How Authentication Works
      2. Pre-Shared Key Authentication
        1. Puppet Setup
      3. SSL Authentication
        1. Server Configuration
          1. Installing and Synchronizing with Puppet
        2. Client Configuration
          1. Create a Client Identity
        3. Key Synchronization
      4. RSA Authentication AES Encryption
        1. Server Configuration
          1. Puppet Module
          2. Manual Config
        2. Client Configuration
        3. Key Synchronization
      5. SSHKey Authentication
        1. Puppet
      6. Authorization
        1. Caller IDs
        2. Defining ActionPolicy with Puppet
          1. Creating Policy in Hiera
          2. Distributing Policy Files
        3. Defining ActionPolicy Manually
      7. Auditing
      8. Conclusion
    5. 12. Challenges of Worldwide Parallelism
  4. III. Plugins
    1. 13. Custom Plugins
      1. Building an Agent
        1. SimpleRPC Framework
        2. Start with a Baseline
        3. Validate Input
        4. Send Replies
        5. Define an Agent DDL
        6. Read Config Files
        7. Install your Agent
        8. Testing the agent
      2. Extending the Agent
        1. Executing Scripts
        2. Executing Commands
        3. Accessing Facts, Agents, and Classes
        4. Results and Exceptions
        5. Logging
      3. Building a Client Application
        1. Baseline Client
        2. Client Filters
        3. Results and Exceptions
        4. Install your Client
      4. Processing Multiple Actions
      5. Creating a Standalone Client
      6. Creating Other Types of Plugins
        1. Authorization Plugins go in util/ directory
        2. Facts Plugins
    2. 14. Registration
      1. Registration Agent
      2. Registration Collector
      3. Registration & SSL Security
    3. 15. Collecting Responses
      1. Create a Listener
      2. Submit Reply-To
      3. Process Responses
    4. You Hold The Strings Now
    5. A. Appendix
      1. Using r10k to install Puppet Modules
      2. Using the PuppetLabs MCollective Module
      3. ActiveMQ Complete Configuration
      4. Using RabbitMQ
        1. Installing RabbitMQ without Puppet
          1. Enable the STOMP Connector and Management Plugins
          2. Install the CLI Tool
          3. Configuring the Queues for MCollective
        2. Configuring an Exchange for an RabbitMQ Federation
      5. OS Specifics
        1. Configuring Debian and Ubuntu Firewalls
        2. FreeBSD
          1. Using the Next Gen Package Manager
          2. Configuring ActiveMQ
          3. Configuring the Firewall
          4. Building Agents from Source
        3. Mac OS X
          1. Installing Ruby
          2. Installing MCollective
        4. Solaris
          1. Installing on Solaris 11
          2. Installing on Solaris 10 and before
        5. Windows
          1. Acquiring Ruby
          2. Adding the RubyGem Dependencies
          3. Installing MCollective
        6. Managing Ruby versions with RVM
  5. About the Author
  6. Copyright