Puppet Types and Providers

Chapter 4. Advanced Types and Providers

The previous chapters covered the core concepts required to write fully functional types and providers. In this chapter, we will explore several advanced features and their implementation. These topics are not required for all custom types, but they have been included to ensure users have a more complete understanding of how Puppet works. Many of Puppet’s native types provide the following functionalities and this chapter will cover how to implement them:

• Resources can respond to refresh events triggered by the notify/subscribe metaparameters.
• Providers may indicate they only support a subset of the functionality of a type interface.
• Types can customize the output of event messages.

The chapter also discusses how code can be shared between multiple providers using both provider inheritance as well as common shared libraries. Code reuse can simplify providers, and reduce the total amount of code that needs to be written and maintained, especially when you need multiple related providers for the same service.

After reading this chapter, you should be able to understand and implement:

• Supporting refresh signals initiated from the subscribe/notify metaparameter
• How providers can support a subset of a type’s features
• Code reuse through parent providers and shared libraries
• Modifying event log messages

Refresh

In Puppet, when any properties are updated, an event is recorded which can trigger updates to other resources with a refresh signal. These special relationships are defined with the notify and subscribe metaparameters. This adds a refresh dependency between resources in addition to a regular order dependency (notify implies before, and subscribe implies require). This section will discuss how to implement the refresh method so a resource can respond to a refresh signal.

The most common usage of refresh relationships in Puppet is to trigger service restarts. When updating application settings, configuration file changes often require service restarts. The following demonstrates how an sshd custom_service can subscribe to changes in its configuration file:

file { '/etc/sshd.conf':
  content => template('ssh/sshd.conf.erb'),
}
custom_service { 'sshd':
  ensure => running,
  subscribe => File['/etc/sshd.conf'],
}

The sshd custom service in this example receives a refresh signal for any changes to the /etc/sshd.conf file. Its type needs to implement the refresh method to respond to these signals, or any refresh signal will simply be ignored. In this case, notify or subscribe simply indicate an ordering relationship and the refresh signal is ignored.

The example below implements the refresh method for the custom_service type. This method instructs the type to call the current provider’s restart method when it receives a refresh signal and the ensure state of the resource was specified as :running:

Puppet::Type.newtype(:custom_service) do
  ...
  def refresh
    if (@parameters[:ensure] == :running)
      provider.restart
    else
      debug "Skipping restart; service is not running"
    end
  end
end

We also need to implement the provider’s restart method invoked by the type’s refresh method:

Puppet::Type.type(:custom_service).provide('service') do

  commands :service => 'service'
  ...
  def restart
    service(resource[:name], 'restart')
  end
  ...
end

Now, custom_service resources will be restarted when they receive refresh signals. The next section will discuss how to create providers that only support a subset of the functionality of its type using the features method.

Features

A single resource type can have multiple provider backends. In some cases, a provider may not support all functionalities described in the resource type. The features method allows the type to specify properties that will only be implemented by a subset of its providers. The providers can ignore feature specific properties unless they offer management for those functionalities and declare support for them. Unlike properties, parameters do not need to label feature support, since providers that do not support a parameter can simply ignore them.

For example, a database resource may have both MySQL and PostgreSQL backends. MySQL tables have the option of selecting a storage engine such as MyISAM, InnoDB, and memory (among several other choices). PostgreSQL does not offer this option since it only offers the built-in storage engine. In this case, the storage engine attribute should be labeled as a feature since it is only supported by one of the products. A single resource type can have multiple provider backends. In some cases, a provider does not support all functionalities described in the resource type. For example, a database resource may have both MySQL and PostgreSQL backend. In this case, the storage engine attribute should be labeled as a feature since it is only supported by one of the products. The features method allows the type to specify properties that require a unique functionality. The providers can ignore feature specific properties unless they support management for those functionalities and declare support for them. Unlike properties, parameters do not need to label feature support, since providers that do not support a parameter can simply ignore them.

A type declares the list of optional functionalities using the feature method with the following three arguments:

1. The name of the feature
2. Documentation for the feature
3. A list of methods a provider should implement to support a feature

The syntax for creating a feature is shown below:

feature :feature_name, "documentation on feature.",
  :methods => [:additional_method]

In our custom_package type from the last chapter, we implemented a property called version. This property is only supported by the subset of providers that have a notion of package versions. The following example demonstrates how a feature, :versionable, can be added to our custom_package type, and how our version property can indicate that it is only supported by providers that are versionable:

Puppet::Type.newtype(:custom_package) do
  ...
  feature :versionable, "Package manager interrogate and return software
    version."

  newproperty(:version, :required_features => :versionable) do
     ...
  end
end

Note that we did not specify a list of methods that are implemented by a provider to indicate that it supports this feature. When no methods are listed, a provider must explicitly declare its support for its feature with the has_feature method:

Puppet::Type.type(:custom_package).provide('yum') do
  has_feature :versionable
end

For custom_package providers that do not support versions, simply omit has_feature :versionable, and the property can be safely ignored. When Puppet encounters providers that do not support a specific feature or providers that are missing the required methods for a feature, it skips properties that depend on those features.

Code Reuse

There are a few ways in which common code can be shared between providers. Sharing code between providers is extremely useful because it reduces duplicate code across all providers. This section will discuss how providers can reuse code from parent providers and shared utility libraries.

Parent Providers

It is possible for multiple providers to use the same commands to perform a subset of their functionality. Providers are allowed a single parent provider. Providers reuse their parent’s methods by default, and can optionally implement methods to override the parent’s behavior.

A provider sets its parent by passing the :parent option to the provide method. The following trivial example shows how a Puppet Enterprise gem provider could reuse all of the existing functionality of the current gem provider and just update the path of the gem executable:

Puppet::Type.type(:package).provide :pe_gem, :parent => :gem do

  commands :gemcmd => "/opt/puppet/bin/gem"
end

The yum and rpm providers that we crafted in the last chapter can use provider inheritance to share most of their functionality. Since the yum provider relies on rpm for retrieving the current state of packages on the system, it can use inheritance to avoid having to reimplement these methods. The following example is the rpm provider which will be the parent provider for yum:

Puppet::Type.type(:custom_package).provide(:rpm) do
  commands :rpm => 'rpm'
  mkresource_method

  self.prefetch
    packages = rpm('-qa','--qf','%{NAME} %{VERSION}-%{RELEASE}\n')
    packages.split("\n").collect do |line|
      name, version = line.split
      new( :name => name,
           :ensure => :present,
           :version => version,
      )
    end
  end

  self.instances
    packages = instances
    resources.keys.each do |name|
      if provider = packages.find{ |pkg| pkg.name == name }
        resources[name].provider = provider
      end
    end
  end

  def exists?
    @property_hash[:ensure] == :present
  end

  def create
    fail "RPM packages require source parameter" unless resource[:source]
    rpm('-iU', resource[:source])
    @property_hash[:ensure] = :present
  end

  def destroy
    rpm('-e', resource[:name])
    @property_hash[:ensure] = :absent
  end
end

The provider above already implements several of the exact methods that our yum provider needs, namely: self.instances, self.prefetch, and exists?. The example below demonstrates how our yum provider can set its parent to the rpm provider and override that provider’s create and destroy methods:

Puppet::Type.type(:custom_package).provide(:yum, :parent => :rpm) do
  commands :yum => 'yum'
  commands :rpm => 'rpm'

  def create
    if resource[:version]
     yum('install', '-y', "#{resource[:name]}-#{resource[:version]}")
    else
      yum('install', '-y', resoure[:name])
    end
    @property_hash[:ensure] = :present
  end

  def destroy
    yum('erase', resource[:name])
    @property_hash[:ensure] = :absent
  end
end

Note

A child provider does not currently share commands with its parent provider. Commands specified in the parent need to be specified again in the child using the commands methods.

Ruby extensions can share common code without using parent providers. Types and Providers occasionally need to share common libraries. The next section will discuss the conventions and challenges with sharing common code in custom Ruby extensions.

Shared Libraries

Puppet Labs recommends that utility code located in modules be stored in the following namespace: lib/puppet_x/<organization>/. Utility code should never be stored in lib/puppet because this may lead to unintended conflicts with puppet’s source code, or with the source code from other providers.

The following directory tree contains an example module with two types, each of which has one provider. It also contains a class with some helper methods.

`-- lib
    |-- puppet
    |   |-- provider
    |   |   `-- one
    |   |       `-- default.rb
    |   |   `-- two
    |   |       `-- default.rb
    |   `-- type
    |       |-- one.rb
    |       `-- two.rb
    `-- puppet_x
        `-- bodeco
            `-- helper.rb

Note

For more information on this convention, see its Puppet Labs project issue, #14149.

Let’s create a helper method shared among both providers:

class Puppet::Puppet_X::Bodeco::Helper
  def self.make_my_life_easier
  ...
  end
end

All code in a module’s lib directory is pluginsynced to agents along with types and providers. This does not, however, mean that all Ruby code in a module’s lib directory will automatically be available in Ruby’s LOADPATH.

Due to limitations around how Puppet currently handles Ruby libraries, code should only be shared within the same module, and then it should only be used by requiring the relative path to the file. The provider should require the library as follows:

require File.expandpath(File.join(File.dirname(__FILE__), '..', '..', , '..', 'puppet_x', 'bodeco', 'helper.rb'))
Puppet::Type.type(:one).provide(:default) do

  def exists?
    Puppet::Puppet_X::Helper.make_my_life_easier
  end

end

The require method above should be explained in a little more detail:

1. FILE provides the full path of the current file being processed.
2. File.dirname is invoked in the full path of this file to return its directory name.
3. File.join is used to append the relative path ../../../puppet_x/bodeco to our current directory path.
4. File.expand_path is used to convert the relative path into an absolute path.

The result of these methods is a relative path lookup for the helper utility in our current module. This relative path lookup is not recommended across modules, since modules can exist in different module directories that are both part of the current modulepath.

Customizing Event Output

Whenever Puppet modifies a resource, an event is recorded. The event message can be customized per resource attribute by overriding the should_to_s, is_to_s, and change_to_s methods.

When executing Puppet, if the current state of the resource does not match the resource specified desired state, Puppet will display the following log message:

notice: /#{resource_type}[#{resource_title}]/#{resource_attribute}:
  current_value 'existing_value', should be 'desired_value' (noop)

The output displayed for the current value is determined by calling is_to_s on the retrieved value of the resource. The value for the desired value is determined by calling should_to_s on the munged property value.

By default, Puppet simply transforms the attribute value to a string with Ruby’s built-in method to_s. For hash values, this results in an incomprehensible string output. The following irb snippet shows what happens when you call to_s on a hash:

>> {'hello'=>'world'}.to_s
=> "helloworld"

If the property returns this hash value, the Puppet notice message would be "should be 'helloworld’”. We can use the should_to_s and is_to_s methods as follows to override how hashes are displayed in Puppet’s output:

  newproperty(:my_hash) do
    def should_to_s(value)
       value.inspect
     end

     def is_to_s(value)
       value.inspect
     end
   end

Now when the resource changes, the message is much more readable:

notice: ... : current_value '{"hello"=>"world"}', should be
  '{"goodbye"=>"world"}'

Usually, updating these methods to .inspect will provide sufficiently readable output, but in some cases where the attribute contains a long list of array values, it’s helpful to display the differences rather than list all values. In these situations, the change_to_s method provides the flexibility to format this output:

  newproperty(:my_array) do
    def change_to_s(current, desire)
      "removing #{(current-desire).inspect},
        adding #{(desire-current).inspect}."
    end
  end

For hashes, there’s rubygems hashdiff, which will show the differences between two hashes:

require 'rubygems'
require 'hashdiff'

  newproperty(:my_array) do
    def change_to_s(current, desire)
      "removing #{(HashDiff.diff(current,desire).inspect},
        adding #{HashDiff.diff(desire, current).inspect}."
    end
  end

Now What?

This book covered the types and providers APIs used to implement custom resources. With this knowledge, you should understand when and why—as well as how—to write native resource types. We certainly have not explored every possible API call used by Puppet’s native types. Some were ignored on purpose because they are fairly complex and we do not advocate using them, while others were omitted because the value of using them is not clear, even to us.

For the more adventurous readers, the Puppet source code contains examples of every possible supported API call: lib/puppet/{type,provider,property}.rb. In fact, we often used Puppet’s source code as a reference to ensure that concepts were correctly explained for this book.

For new Puppet developers, the following resources are available for continued assistance:

• The google puppet-dev mailing list
• The Freenode IRC channels #puppet and #puppet-dev