Chapter 4. Agents
The server daemon
mcollectived can only respond to ping and inventory requests, which isn’t all that useful. It is not inappropriate to consider
mcollectived to be a a container for
agents. We will install several
agent plugins in this chapter to extend and enhance what we can control on the server.
Each agent has a matching
application which knows how to issue requests specific to that agent. The agent will respond to requests coming from the matching application.
Puppet Labs provides a number of MCollective agents that know how to do common systems management task: query, start, and stop processes; query, install, and remove packages. We’ll start with just the plugins they provide for now.
However, you may have needs which are custom and unique to your environment. MCollective is extensible, and you’ll find many user-provided agents available on the Puppet Forge. You will also find it easy to build your own agents and clients. In those situations you can orchestrate things that the MCollective maintainers never dreamed about. We’re going to show you how to do this in Part III.
On each server in your environment we have installed the mcollectived service. For this daemon to operate correctly it requires two plugins:
connector pluginto establish a link with the middleware and subscribe to topics
security pluginto encrypt and decrypt the communications
These two connectors must be the same throughout your environment. In most situations, the configuration for these plugins will be the same on every server host.
For the baseline setup described in Chapter 2 we used:
The alternative would have been to use RabbitMQ, or to build a custom middleware connector.
We will discuss alternate security plugins like SSL Security in Chapter 11.
Installing from Packages
If you are using the Puppet Labs repositories as described in Puppet Labs Repository, you can simply install the baseline set of agents as follows:
sudo yum install mcollective-filemgr-agent$
sudo yum install mcollective-nettest-agent$
sudo yum install mcollective-package-agent$
sudo yum install mcollective-service-agent
sudo apt-get install mcollective-filemgr-agent$
sudo apt-get install mcollective-nettest-agent$
sudo apt-get install mcollective-package-agent$
sudo apt-get install mcollective-service-agent
You’ll need to do this on every server in your environment. On the client nodes you’ll need to install the corresponding client package, for example:
sudo yum install mcollective-filemgr-client
For any other operating systems you should install from the operating system repository or from source, as described in the next section.
Installing from Source
Installing mcollective plugins from source is actually quite easy, and is exactly the same on every operating system I’ve tested the process on. You’ll need to have
git installed and working on your host.
git clone git://github.com/puppetlabs/mcollective-Cloning into 'mcollective-filemgr-agent'... remote: Reusing existing pack: 49, done. remote: Total 49 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (49/49), done. Checking connectivity... done $
mco plugin package .This will build agent, client, and common packages for many platforms.
If MCollective does not know how to build packages for your operating system yet, then you’ll need to copy the files into their location by hand, or with a configuration tool.
Copy Agents into Location
MCollective plugins are expected to mimic the structure of the libdir/mcollective directory. This means that the plugin will have an agent directory for the agent plugin, and an application directory for the client. You can ignore any Rakefiles or spec directories in the plugin.
All MCollective plugins must be copied into the libdir, as specified in the client and server config files. libdir should be the standard Ruby lib directory containing an mcollective directory, under which lies an agent directory for the agent plugins, and an application directory for the client.
In short: agent plugin files are named libdir/mcollective/agent/NAME.(rb|ddl|erb). There is usually a client application in libdir/mcollective/agent/NAME.rb. There may be util or other directories, which should be copied verbatim.
For example, here are the agent files installed for the
On Red Hat-like systems:
On Debian-like systems:
On other platforms:
You’ll probably want to use configuration management to deploy these files to all of your servers.
The mcollective directory goes inside libdir. In the Red Hat case, this means the complete path contains the string
mcollective/mcollective; be careful not to accidentally skip the second mcollective.
More details are available at http://docs.puppetlabs.com/mcollective/deploy/plugins.html.
After you have installed new agents on a server node you tell
mcollectived to reload the agents. The simplest method is to restart
mcollectived for the agents to be loaded and active.
sudo service mcollective restartShutting down mcollective: [ OK ] Starting mcollective: [ OK ]
You can also send the daemon a USR1 signal to effect the reload. On many platforms you can do this with
-x option can be used to ensure you don’t kill any other program with a partial name match.
sudo pkill -USR1 -x mcollectived || echo "pkill failed: $?"
Once you have made the changes, you can use the
inventory request and read through the output to see if the new agents are available on the node:
mco inventoryread down through the output... Agents: discovery rpcutil
filemgr nettest package service
I rather like using
awk to remove all other output. Naturally it’s not guaranteed against future changes in the output.
nodename| awk '/Agents:/','/^$/'
You can also query to get a list of every server which has the agent installed:
mco find --with-agentgeode heliotrope sunstone
To interact with these agents we need to have installed the client plugins on any system from which we will be sending requests to these agents, as discussed in Using Client Plugins.
I have seen inconsistent response when using the
USR1 signal to reload the agent. It seems to work most times for making newly added agents available, but is less consistent when reloading an upgraded agent. If you don’t see what you expect, it is best to perform a full restart of
What if you want to disable an agent on a machines? For example, if there was a problem with the agent but you didn’t want to go through uninstalling it yet.
There are two ways to disable an agent. The first option is within the server configuration file:
plugin_name.activate_agent = false
The other way is to create a configuration file for that particular agent:
$ echo "activate_agent = false" | sudo tee -a /etc/mcollective/plugins.d/
Any time this parameter is changed you’ll need to signal
mcollectived to reload as documented in Notify
Using Client Plugins
Client plugins provide an
application to issue commands to their respective agents. They are not useful until you install the matching
server agent as described in Agent Plugins.
If you are using the Puppet Labs repositories as described in Puppet Labs Repository, you can install a baseline set of safe and well-tested clients like this:
# RedHat, CentOS, Fedora Based Systems $
sudo yum install mcollective-filemgr-client$
sudo yum install mcollective-nettest-client$
sudo yum install mcollective-package-client$
sudo yum install mcollective-service-client# Debian or Ubuntu $
sudo apt-get install mcollective-filemgr-client$
sudo apt-get install mcollective-nettest-client$
sudo apt-get install mcollective-package-client$
sudo apt-get install mcollective-service-client
After you install the client plugins, you can get a list of
applications available on a node with the
mco plugin docPlease specify a plugin. Available plugins are: Agents: filemgr File Manager nettest Agent to do network tests from a mcollective host package Install and uninstall software packages ...etc
The applications add custom subcommands (called faces) to the
mco client, allowing easy access to the commands provided by each client plugin. You can get documentation for how to use the plugin from the
mco help packageInstall, uninstall, update, purge and perform other actions to packages Usage: mco package [OPTIONS] [FILTERS] <ACTION> <PACKAGE> Usage: mco package <PACKAGE> <install|uninstall|purge|update|status> The ACTION can be one of the following: install - install PACKAGE uninstall - uninstall PACKAGE purge - uninstall PACKAGE and purge related config files update - update PACKAGE status - determine whether PACKAGE is installed and report its version $
mco package update mcollective -I* [ ==================================================> ] 1 / 1 Summary of Ensure: 2.5.0-1.el6 = 1 Finished processing 1 / 1 hosts in 5923.31 ms
You’ll find the
help command useful for determining command syntax, but extensive details about inputs and output can only be found in the plugin’s documentation.
mco plugin doc agent/package ======= Install and uninstall software packages Author: R.I.Pienaar Version: 4.2.2 License: ASL 2.0 Timeout: 180 Home Page: https://github.com/puppetlabs/mcollective-package-agent Requires MCollective 2.2.1 or newer ACTIONS: ======== apt_checkupdates, apt_update, checkupdates, install, purge, status, uninstall, update, yum_checkupdates, yum_clean apt_checkupdates action: ------------------------ Check for APT updates INPUT: This action does not have any inputs OUTPUT: exitcode: Description: The exitcode from the apt command Display As: Exit Code ...etc
Finding Community Plugins
Before you set out to build your own plugin, you should look around to ensure that nobody hasn’t built it already. There are many benefits to using something which is easy to extend. One of them is that it builds a large community of people who extend the software to meet their own needs. The other is that the community can share the plugins with each other. So before you go off to build a module yourself, you should check to make sure someone hasn’t already made an agent for this.
The first location to look for plugin agents and clients should always be the Puppet Labs Yum or APT repositories. This repo and how to use it is documented in Puppet Labs Repository.
yum search --enablerepo=puppetlabs*# RedHat $
apt-cache search# Debian
Puppet Labs maintains a wiki page with links to many of the plugins at http://projects.puppetlabs.com/projects/mcollective-plugins/wiki
At the time this book was written the only place where the Author saw significant community MCollective plugin development was on GitHub. GitHub contains hundreds of MCollective plugins that others have already developed. The following query will find most of the plugins available for download: GitHub Search mcollectivel
Unfortunately, many of the modules available on GitHub are made for older versions of MCollective. It can be difficult to sift through everything to find the gems. As the requirements for plugins have changed significantly in the last two years, I would recommend avoiding any plugin which hasn’t been updated in 2013 or later.
It is always best if you can build packages of the plugins with the
mco plugin package command. Install the
-agent packages on servers, and the
-client packages on clients. You’ll need to follow the steps outlined in Notify
mcollectived so that the server will pick up the new agents.
If you can’t build a package with the plugin, the process for installing plugins from source is documented in Installing from Source.