Chapter 5. Agent and Client Plugins

The software installed on the nodes you control through MCollective is a daemon called mcollectived. We will install several agents in this chapter to provide powerful new features. mcollectived is called a server because it functions as an application server. Its abilities are expanded by installing agent plugins to extend and enhance what we can control on the node.

Each agent has a matching client or application that knows how to issue requests specific to that agent. We’ll install and use the client applications to communicate with the agent.

Connector Plugins

On each node in your environment, we have installed the mcollectived service. For this daemon to operate correctly, it requires two plugins:

  • A connector plugin to establish a link with the middleware and subscribe to topics

  • A security plugin to encrypt and decrypt the communications

These two connectors must be the same throughout your environment. In most situations, the configuration for these plugins will be the same for every server.

For the baseline setup described in Chapter 2, we used:

connector = activemq

The alternative would have been to use RabbitMQ or to build a custom middleware connector.

securityprovider = psk

We will discuss alternative security plugins like SSL in Chapter 13.

Installing Agents from Packages

Puppet Labs provides a number of MCollective agents that know how to do common systems-management tasks (e.g., query, start, and stop processes, and query, install, and remove packages). We’ll start with just the plugins they provide for now.  

If you are using the Puppet Labs repositories as described in “Puppet Labs Repository”, you can simply install the baseline set of agents as follows:

# for RedHat, CentOS, and Fedora-based nodes
$ sudo yum install mcollective-filemgr-agent
$ sudo yum install mcollective-nettest-agent
$ sudo yum install mcollective-package-agent
$ sudo yum install mcollective-service-agent

# for Debian and Ubuntu hosts
$ sudo apt-get install mcollective-filemgr-agent
$ sudo apt-get install mcollective-nettest-agent
$ sudo apt-get install mcollective-package-agent
$ sudo apt-get install mcollective-service-agent

You’ll need to do this on every server in your environment. On the client nodes, you’ll need to install the corresponding client package, for example:

$ sudo yum install mcollective-filemgr-client

For any other operating systems, you should install from the operating system repository or from source, as described in the next section.

Installing Agents from Source

Installing mcollective plugins from source is actually quite easy and is exactly the same on every operating system I’ve tested the process on. You’ll need to have git installed and working on your host to perform these steps: 

$ git clone git://github.com/puppetlabs/mcollective-filemgr-agent.git
Cloning into 'mcollective-filemgr-agent'...
remote: Reusing existing pack: 49, done.
remote: Total 49 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (49/49), done.
Checking connectivity... done
$ cd mcollective-filemgr-agent
$ mco plugin package .
Building packages for mcollective-filemgr plugin.
Completed building all packages for mcollective-filemgr plugin.

If MCollective does not know how to build packages for your operating system yet, then you’ll need to copy the files into the MCollective plugins directory.

Copy to Plugins Directory

MCollective plugins are expected to mimic the structure of the libdir/mcollective directory. This means that the plugin will have an agent directory for the agent plugin and an application directory for the client. You can ignore any Rakefiles or spec directories in the plugin.

All MCollective plugins must be copied into the libdir, as specified in the client and server config files. libdir should be the standard Ruby lib directory containing an MCollective directory, under which lies an agent directory for the agent plugins and an application directory for the client.

In short: agent plugin files are named libdir/mcollective/agent/NAME.(rb|ddl|erb). There is usually a client application in libdir/mcollective/agent/NAME.rb. There may be util or other directories, which should be copied verbatim.

For example, here are the agent files installed for the package plugin:

For RedHat, CentOS, and Fedora-based systems:

$ cp agent/package.rb /usr/libexec/mcollective/mcollective/agent/
$ cp agent/package.dll /usr/libexec/mcollective/mcollective/agent/
$ cp application/package.rb /usr/libexec/mcollective/mcollective/application/

For Debian and Ubuntu systems, use:

$ cp agent/package.rb /usr/share/mcollective/plugins/mcollective/agent/
$ cp agent/package.dll /usr/share/mcollective/plugins/mcollective/agent/
$ cp application/package.rb /usr/share/mcollective/plugins/mcollective/application/

And for other platforms, use:

$ cp agent/package.rb /opt/mcollective/plugins/mcollective/agent/
$ cp agent/package.dll /opt/mcollective/plugins/mcollective/agent/
$ cp application/package.rb /opt/mcollective/plugins/mcollective/application/

You’ll probably want to use configuration management to deploy these files to all of your servers.

Note

The mcollective directory goes inside libdir. In the Red Hat case, this means the complete path contains the string mcollective/mcollective; be careful not to accidentally skip the second mcollective.

More details are available at http://bit.ly/WBy7RV.

Notify mcollectived

After you have installed new agents on a server node, you tell mcollectived to reload the agents. The simplest method is to restart mcollectived for the agents to be loaded and active: 

$ sudo service mcollective restart
Shutting down mcollective:                                 [  OK  ]
Starting mcollective:                                      [  OK  ]

You can also send the daemon a USR1 signal to effect the reload. On many platforms, you can do this with pkill. The -x option can be used to ensure you don’t kill any other program with a partial name match:

$ sudo pkill -USR1 -x mcollectived || echo "pkill failed: $?"

Once you have made the changes, you can use the inventory request and read through the output to see if the new agents are available on the node:

$ mco inventory nodename
read down through the output...
Agents:
      discovery       rpcutil
      filemgr         nettest
      package         service

I rather like using awk to remove all other output (naturally, it’s not guaranteed against future changes in the output):

$ mco inventory nodename | awk '/Agents:/','/^$/'

You can also query to get a list of every server that has the agent installed:

$ mco find --with-agent filemgr
geode
heliotrope
sunstone

To interact with these agents, we need to have installed the client plugins on any system from which we will be sending requests to these agents, as discussed in “Using Client Plugins”.

Note

I have seen inconsistent response when using the USR1 signal to reload the agent. It seems to work most times for making newly added agents available, but is less consistent when reloading an upgraded agent. If you don’t see what you expect, it is best to perform a full restart of mcollectived.

Disabling Agents

What if you want to disable an agent on a machine? For example, if there was a problem with the agent but you didn’t want to go through uninstalling it yet.

There are two ways to disable an agent. The first option is within the server configuration file:

plugin.plugin_name.activate_agent = false

The other way is to create a configuration file for that particular agent:

$ echo "activate_agent = false" 
  | sudo tee -a /etc/mcollective/plugins.d/plugin_name.cfg
Note

When using the Puppet Enterprise, product changes to the main server config file are not supported. You must create plugin configuration files in /etc/puppetlabs/mcollective/plugin.d/.

Any time this parameter is changed, you’ll need to signal mcollectived to reload as documented in “Notify mcollectived”.

Using Client Plugins

Client plugins provide an application to issue commands to their matching agents. They are only useful when you install the corresponding agent on the servers. 

If you are using the Puppet Labs repositories (as described in “Puppet Labs Repository”), you can install a baseline set of safe and well-tested clients like this:

# RedHat, CentOS, Fedora Based Systems
$ sudo yum install mcollective-filemgr-client
$ sudo yum install mcollective-nettest-client
$ sudo yum install mcollective-package-client
$ sudo yum install mcollective-service-client

# Debian or Ubuntu
$ sudo apt-get install mcollective-filemgr-client
$ sudo apt-get install mcollective-nettest-client
$ sudo apt-get install mcollective-package-client
$ sudo apt-get install mcollective-service-client

After you install the client plugins, you can get a list of applications available on a node with the doc command:

$ mco plugin doc
Please specify a plugin. Available plugins are:

Agents:
  filemgr                   File Manager
  nettest                   Agent to do network tests from a mcollective host
  package                   Install and uninstall software packages
...etc

The applications add custom subcommands (called faces) to the mco client, allowing easy access to the commands provided by each client plugin. You can get documentation for how to use the plugin from the help command:

$ mco help package

Install, uninstall, update, purge and perform other actions to packages

Usage: mco package [OPTIONS] [FILTERS] <ACTION> <PACKAGE>
Usage: mco package <PACKAGE> <install|uninstall|purge|update|status>

The ACTION can be one of the following:

    install    - install PACKAGE
    uninstall  - uninstall PACKAGE
    purge      - uninstall PACKAGE and purge related config files
    update     - update PACKAGE
    status     - determine whether PACKAGE is installed and report its version

$ mco package update mcollective -I heliotrope
 * [ ==================================================> ] 1 / 1

Summary of Ensure:
   2.5.3-1.el6 = 1

Finished processing 1 / 1 hosts in 5923.31 ms

You’ll find the help command useful for determining command syntax, but extensive details about inputs and outputs can be found only in the plugin’s documentation: 

$ mco plugin doc agent/package
package
=======

Install and uninstall software packages

      Author: R.I.Pienaar
     Version: 4.3.1
     License: ASL 2.0
     Timeout: 180
   Home Page: https://github.com/puppetlabs/mcollective-package-agent

   Requires MCollective 2.2.1 or newer

ACTIONS:
========
   apt_checkupdates, apt_update, checkupdates, install, purge, status, uninstall,
   update, yum_checkupdates, yum_clean

   apt_checkupdates action:
   ------------------------
       Check for APT updates

       INPUT:
          This action does not have any inputs

       OUTPUT:
           exitcode:
              Description: The exitcode from the apt command
               Display As: Exit Code
...etc

Finding Community Plugins

You may have needs that are custom and unique to your environment. MCollective is extensible, and you’ll find many user-provided agents available on the Puppet Forge. You will also find it easy to build your own agents and clients. In those situations, you can orchestrate actions that the MCollective maintainers never dreamed about. I’ll show you how to do this in Part III.

Before you set out to build your own plugin, you should look around to ensure that someone hasn’t built it already. There are many benefits to using something that is easy to extend. One of them is that it builds a large community of people who extend the software to meet their own needs. The other is that members of the community can share the plugins with one another. So before you go off to build a module yourself, you should check to make sure someone hasn’t already made an agent for this.

The first location to look for plugin agents and clients should always be the Puppet Labs Yum or APT repositories (this repo and how to use it is documented in “Puppet Labs Repository”):

$ yum search --enablerepo=puppetlabs* mcollective # RedHat
$ apt-cache search mcollective                    # Debian

Puppet Labs maintains a wiki page with links to many of the plugins.

At the time this book was written, the only place where I saw significant community MCollective plugin development was on GitHub. GitHub contains hundreds of MCollective plugins that others have already developed. Running a GitHub search for “mcollective” will return most of the plugins available for download.

Warning

Unfortunately, many of the modules available on GitHub are made for older versions of MCollective. It can be difficult to sift through everything to find the gems. As the requirements for plugins have changed significantly in the last two years, I would recommend avoiding any plugin that hasn’t been updated in 2013 or later.

It is always best if you can build packages of the plugins with the mco plugin package command. Install the -agent packages on servers and the -client packages on clients. You’ll need to follow the steps outlined in “Notify mcollectived” so that the server will pick up the new agents.

If you can’t build a package with the plugin, the process for installing plugins from source is documented in “Installing from Source”.

Get Learning MCollective now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial