Managing Infrastructure with Puppet

Getting Started

Once Puppet is installed, you will have the puppet command at your disposal. The first thing you should do is run puppet describe --list. This will provide a list of the available resource “types” you have to work with out of the box:

:> puppet describe --list
    These are the types known to puppet:
augeas          - Apply the changes (single or array of changes ...
computer        - Computer object management using DirectorySer ...
cron            - Installs and manages cron jobs
exec            - Executes external commands
file            - Manages local files, including setting owners ...
filebucket      - A repository for backing up files
group           - Manage groups
host            - Installs and manages host entries
k5login         - Manage the `
macauthorization - Manage the Mac OS X authorization database
mailalias       - Creates an email alias in the local alias dat ...
maillist        - Manage email lists
mcx             - MCX object management using DirectoryService  ...
mount           - Manages mounted filesystems, including puttin ...
nagios_command  - The Nagios type command
nagios_contact  - The Nagios type contact
nagios_contactgroup - The Nagios type contactgroup
nagios_host     - The Nagios type host
nagios_hostdependency - The Nagios type hostdependency
nagios_hostescalation - The Nagios type hostescalation
nagios_hostextinfo - The Nagios type hostextinfo
nagios_hostgroup - The Nagios type hostgroup
nagios_service  - The Nagios type service
nagios_servicedependency - The Nagios type servicedependency
nagios_serviceescalation - The Nagios type serviceescalation
nagios_serviceextinfo - The Nagios type serviceextinfo
nagios_servicegroup - The Nagios type servicegroup
nagios_timeperiod - The Nagios type timeperiod
notify          - Sends an arbitrary message to the agent run-t ...
package         - Manage packages
resources       - This is a metatype that can manage other reso ...
schedule        - Defined schedules for Puppet
selboolean      - Manages SELinux booleans on systems with SELi ...
selmodule       - Manages loading and unloading of SELinux poli ...
service         - Manage running services
ssh_authorized_key - Manages SSH authorized keys
sshkey          - Installs and manages ssh host keys
stage           - A resource type for specifying run stages
tidy            - Remove unwanted files based on specific crite ...
user            - Manage users
whit            - The smallest possible resource type, for when ...
yumrepo         - The client-side description of a yum reposito ...
zfs             - Manage zfs
zone            - Solaris zones
zpool           - Manage zpools

We’ll primarily be concerned with the file, exec, cron, user, group, and package types. In addition to these built-in types, a large variety of user-contributed modules add functionality for nearly every commonly used configuration scenario. Documentation of the built-in types can be found on the Puppet Labs documentation site at http://docs.puppetlabs.com/references/2.6.0/type.html.

To get some detail about each of these resource types, you can use puppet describe type. This will output Puppet’s documentation on that particular resource type including parameters and often usage examples as well:

:> puppet describe host

host
====
Installs and manages host entries.  For most systems, these
entries will just be in `/etc/hosts`, but some systems (notably OS X)
will have different solutions.


Parameters
----------

- **ensure**
    The basic property that the resource should be in.  Valid values are
    `present`, `absent`.

- **host_aliases**
    Any aliases the host might have.  Multiple values must be
    specified as an array.

- **ip**
    The host's IP address, IPv4 or IPv6.

- **name**
    The host name.

- **target**
    The file in which to store service information.  Only used by
    those providers that write to disk.

Providers
---------
parsed

Note

puppet describe type -s will give you a less verbose description. This is useful if you just want to know the correct name of a parameter without having to grep through pages of text.

You can also use Puppet to make queries to the resource abstraction layer and return the current state of things on a system. This makes reproducing a particular configuration on an existing system easy when there is a supported resource type. The command for this is puppet resource type name. Here is an example query using the host resource:

:> puppet resource host

host { 'example.example.com':
    host_aliases => ['example'],
    target => '/etc/hosts',
    ip => '10.0.1.101',
    ensure => 'present'
}
host { 'localhost':
    target => '/etc/hosts',
    ip => '127.0.0.1',
    ensure => 'present'
}

:> puppet resource host example.example.com

host { 'example.example.com':
    host_aliases => ['example'],
    target => '/etc/hosts',
    ip => '10.0.1.101',
    ensure => 'present'
}

Resource types are the building blocks of Puppet configurations and most of your time will be spent using them or writing new types to suit your needs. Let’s start with a simple declaration of a package resource.

Files and Packages

This first statement declares that the package ntp should be installed and that the file ntp.conf should be defined with the given contents and permissions at the path /etc/ntp.conf, but only after the package ntp is installed. You can go ahead and test this out (on a test system!) by saving the above text to test.pp and executing puppet apply test.pp. When this manifest is run against a blank system, the agent will check for the existence of an ntp package and install it if necessary. Then the file at /etc/ntp.conf will be installed if it doesn’t exist or overwritten with the content specified if it differs:

package { 'ntp': ensure => installed }

file { 'ntp.conf':
    path => '/etc/ntp.conf',
    mode => 640
    content => '
    driftfile /var/lib/ntp/ntp.drift
    statistics loopstats peerstats clockstats
    filegen loopstats file loopstats type day enable
    filegen peerstats file peerstats type day enable
    filegen clockstats file clockstats type day enable
    server 0.pool.ntp.org
    server 1.pool.ntp.org
    restrict -4 default kod notrap nomodify nopeer noquery
    restrict -6 default kod notrap nomodify nopeer noquery
    restrict 127.0.0.1
    restrict ::1 
    ',
    require => Package[ntp],
}

A few notes here about the syntax: The capitalization of type in resources is important. You can see that when the resources file and package are declared, they are not capitalized, but when the file resource references the ntp package, it is capitalized. Always capitalize the first letter in the type when you are referring to a resource that you have declared elsewhere, but do not capitalize the type in the declaration itself. Also notice that the package declaration at the top is a sort of shortened form, leaving out line breaks and the comma at the end of the single parameter. The last comma is optional on a parameter list, but it is generally included in the full form.

The path, mode, and content parameters are fairly mundane, but the require parameter is special magic. The Puppet agent doesn’t have any innate sense of order of execution when it is run on a manifest or set of manifests. Things will happen in random sequence unless constrained by some dependencies. require is one of those dependencies. The above statement specifies that the file definition ntp.conf requires that the package ntp be installed before it is created. Conversely, we could have specified in the package declaration for ntp that it be run before => File['ntp.conf']. Next, we’ll look at a slightly more streamlined implementation:

package { 'ntp': ensure => '1:4.2.6.p2+dfsg-1ubuntu5' }

file { '/etc/ntp.conf':
        mode => '640',
        owner => root,
        group => root,
        source => '/mnt/nfs/configs/ntp.conf',
        require => Package[ntp],
    }

The most obvious change here is that we’ve moved the file content to an external source. We’ve told Puppet to go and look in /mnt/nfs/configs for a file named ntp.conf and put it in /etc/ntp.conf. For the moment, we’ll use an NFS mount to distribute our configuration files. In later examples, we can use Puppet’s built-in artifice for that purpose. It’s good practice to specify both file permissions and ownership in your manifests, as well as package versions. I’ve replaced the ensure value with an explicit ntp package version. Puppet is intended to be used to make configuration changes as well as to ensure the correctness of configurations. You can think of it both as a deployment script and an auditing tool; by being explicit with your definitions, you can be very confident that your deployment will always work the same way. Finally, I’ll note that this file resource lacks an explicit path parameter. This is because, in Puppet, each type has a parameter that defaults to the resource name. This is referred to as the namevar, and for the file type, it is the source.

Services and Subscriptions

Let’s add a watchdog to ensure that the ntp daemon that we’ve installed is actually running. This will give us some insurance that the proper services have been started, but by no means should it be considered a replacement for a service manager daemon.

I’ve added a service definition that subscribes to the ntp package and its configuration file. On execution, this definition will look in the process table for the pattern “ntpd”. If it fails to find a match for the pattern, Puppet will start the ntp service to ensure that it is running. It also holds a subscription to the ntp package and the file at /etc/ntp.conf. If we later change the config file or update the package version, Puppet will restart the service automatically:

package { 'ntp': ensure => '1:4.2.6.p2+dfsg-1ubuntu5' }

file { '/etc/ntp.conf':
        mode => 640,
        owner => root,
        group => root,
        source => '/mnt/nfs/configs/ntp.conf',
        require => Package[ntp],
        }

service { "ntp":
    ensure => running,
    enable => true,
    pattern => 'ntpd',
    subscribe => [Package["ntp"], File["/etc/ntp.conf"]],
}

Warning

Make sure to test the behavior of the service you are managing. It may be innocuous to restart ntp when the config changes, but it’s an ugly mess when you push a change that, unforeseen, restarts your production database.

2.7 update: In Puppet versions prior to 2.7, the service resource would use a pattern parameter to grep the process table for your running service by default. In 2.7, the optional hasstatus parameter has been changed to default to true. This means that Puppet will assume that the initialization script that is called for your service has a status function that can be called. If you still need to use the process table lookup functionality, you will need to set hasstatus => false.

Exec and Notify

Subscribing a service to a file is very convenient, but what if we need to do something more explicit when a file resource changes? I’ll use a postfix transport map as an example. When this file is updated, I want to run postmap to compile the transport.db file.

In this example, I’ve specified an exec resource. This is the “brute force” resource in Puppet. You can use it to execute commands and shell scripts of your choosing, but there is an important caveat. The command must be idempotent. This means that your system configuration must be able to cope with having the command run over and over again. An exec type resource will generally be run on every Puppet run. The following example specifies that the command should not run unless the subscription to the /etc/postfix/transport file is changed and a refresh is triggered. This is accomplished with the refreshonly parameter. Any exec can be refreshed either by a subscription or a notification. Notification works in the reverse of a subscription:

file { "/etc/postfix/transport":
        mode => 640
        owner => root,
        group => postfix,
        source => '/mnt/postfix/configs/transport',
        }
exec { "postmap /etc/postfix/transport":
        subscribe => File["/etc/postfix/transport"],
        refreshonly => true,
        }

Here we have the file resource notifying the exec of a change. Note that notify implies the behavior that would be seen with a before parameter and subscribe implies the ordering of a require parameter. In this example, the file will be created before the exec is run, and in the former example, the exec requires that the file be run first:

file { "/etc/postfix/transport":
        mode => 640
        owner => root,
        group => postfix,
        source => '/mnt/postfix/configs/transport',
        notify => Exec["postmap /etc/postfix/transport"],
        }
exec { "postmap /etc/postfix/transport":
        refreshonly => true,
        }

There are a couple of scenarios where you might want to use an exec, but only when some other condition requires it. Exec can be used to generate a file; for example, if I wish to fetch a configuration file that I’ve published on a web server.

In the first example, Puppet understands that the result of the exec is to create the file listed in the creates parameter. This exec will only be run if that file doesn’t exist. The second example has the same effect, but it does so using a more customizable condition. The command will only be run if the exit status of the command in the onlyif parameter is zero. Nonzero status will cause the exec to be skipped:

exec { 'curl http://example.com/config/my.conf -o "/etc/myapp/my.conf"':
    creates => "/etc/myapp/my.conf",
    }

exec { 'curl http://example.com/config/my.conf -o "/etc/myapp/my.conf"':
    onlyif => "test ! -e /etc/myapp/my.conf",
    }

Note

Exec is very powerful and it has plenty of appropriate uses. It is not advisable, however, to treat every problem as a potential nail for this particular hammer. An exec is difficult to make platform-agnostic, and it generally solves only one particular problem. In a case where no existing Puppet abstraction does what you need, it might be more useful to dig around in the community modules for an adaptable function. You could even write your own.

Facts, Conditional Statements, and Logging

It’s time to begin talking about what Puppet is doing when it executes these definitions. Each type has a set of “provider” backends that specify what to do with all of the parameters we’ve given it. Each type also has a specified default provider, depending on the nature of the machine you are executing on. In the package definition for ntp we have not told Puppet how to install the package or what commands to use. Instead it knows that we are on an Ubuntu system and has a specified default provider of “apt”. The providers can be explicitly passed in a parameter such as provider => apt,, but this is generally unnecessary and even undesirable. If you were writing Puppet automation for a heterogeneous environment with both CentOS and Ubuntu hosts, it would benefit you to allow Puppet to make the choice.

Note

It’s a great habit to write your manifests to be as operating system independent as you can manage. Not only will it help make your system more versatile, but it will make it convenient for others in the community to reuse when you graciously contribute it back!

This begs the question: How does Puppet know what OS it’s running on? The answer lies with the facter command. Go ahead and execute facter --puppet and inspect the results. You’ll see that Facter knows a lot about your system configuration. Facter comes with a wide range of “facts” defined that describe all different parts of your system. To ascertain what OS it’s running on, Puppet uses the Facter library and looks up the $operatingsystem fact. These facts are also available to us in the manifests themselves. If we would rather make explicit decisions about what to do in different situations (like on different operating systems), we can do that with facts.

In this example, I’ve added a selector operation into the source parameter. This specifies that if the $operatingsystem fact is Ubuntu, we should use the source file at /mnt/nfs/configs/ubuntu-ntp.conf; else we should use the default source file. Classic if-else and case statements are also allowed:

package { 'ntp': ensure => '1:4.2.6.p2+dfsg-1ubuntu5' }

file { '/etc/ntp.conf':
                mode => '640',
                owner => root,
                group => root,
                source => $operatingsystem ? {
                    'Ubuntu' => '/mnt/nfs/configs/ubuntu-ntp.conf',
                    default => '/mnt/nfs/configs/default-ntp.conf',
                                },
                require => Package[ntp],
                        }

service { "ntp":
        ensure => running,
        enable => true,
        pattern => 'ntpd',
subscribe => [Package["ntp"], File["/etc/ntp.conf"]],
    }

Here we’ve made a simple decision tree that prints out a notice depending on the OS type and version reported by Facter. Notices can be useful for logging of Puppet runs and reporting on exceptional conditions. Puppet can be very verbose about what changes it’s made, but custom logging is convenient:

if $operatingsystem == 'Ubuntu' {
    case $operatingsystemrelease {
        '11.04':    { notice("Natty Narwahl") }
        '10.10':    { notice("Maverick Meerkat") }
        '10.04':    { notice("Lucid Lynx") }
        }
} else {
    notice("We're not on Ubuntu!")
}

With these basic tools alone, we have enough to begin writing some convenient system installation scripts. That would let us build up a big manifest full of resource declarations and decision structures and then apply them to a system with Puppet. This manual execution is useful for writing and testing Puppet manifests, but as we’ll see in the next chapter, we can let the servers configure themselves instead.