book

Learning Puppet 4

by Jo Rhett

April 2016

Intermediate to advanced

594 pages

12h 53m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who This Book Is ForWhat to Expect from MeWhat You Will NeedWhat You’ll Find in This BookHow to Use This BookIPv6 ReadySSL is now TLSConventions Used in This BookSafari® Books OnlineHow to Contact UsContent UpdatesFebruary 9, 2018Acknowledgments
What Is Puppet?Why DeclarativeHow Puppet WorksWhy Use PuppetIs Puppet DevOps?Time to Get Started
Handling ChangeUsing IdempotenceDeclaring Final StateReviewing Declarative Programming
Installing VagrantInstalling Vagrant on MacInstalling Git Tools on WindowsInstalling VirtualBox on WindowsInstalling Vagrant on WindowsStarting a Bash ShellDownloading a BoxCloning the Learning RepositoryInstall the Vagrant vbguest PluginInitializing the Vagrant SetupVerifying the /vagrant FilesystemInitializing Non-Vagrant SystemsInstalling Some Helpful UtilitiesChoosing a Text EditorOn the Virtual SystemOn Your DesktopIn Your ProfileReviewing the Learning Environment
Adding the Package RepositoryWhat Is a Puppet Collection?Installing the Puppet AgentReviewing DependenciesReviewing Puppet 4 ChangesLinux and UnixWindowsMaking Tests ConvenientRunning Puppet Without sudoRunning Puppet with sudoReviewing Puppet Installation
Implementing ResourcesApplying a ManifestDeclaring ResourcesViewing ResourcesExecuting ProgramsWas That Idempotent?Managing FilesFinding File BackupsRestoring FilesAvoiding Imperative ManifestsTesting YourselfReviewing Writing Manifests
Defining VariablesDefining NumbersCreating Arrays and HashesMapping Hash Keys and ValuesUsing Variables in StringsRedacting Sensitive ValuesUsing Braces to Limit ProblemsPreventing InterpolationUsing Unicode CharactersAvoiding RedefinitionAvoiding Reserved WordsLearning MoreFinding FactsCalling Functions in ManifestsUsing Variables in ResourcesDefining Attributes with a HashDeclaring Multiple Resource TitlesDeclaring Multiple Resource BodiesModifying with OperatorsAdding to Arrays and HashesRemoving from Arrays and HashesOrder of OperationsUsing Comparison OperatorsEvaluating Conditional ExpressionsMatching Regular ExpressionsBuilding Lambda BlocksLooping Through Iterationseach()filter()map()reduce()slice()with()Capturing Extra ParametersTerminating Iteration EvaluationIteration Wrap-UpReviewing Puppet Configuration Language
Adding AliasesSpecifying an Alias by TitleAdding an Alias MetaparameterPreventing ActionAuditing ChangesDefining Log LevelFiltering with TagsSkipping TagsLimiting to a ScheduleUtilizing periodmatchAvoiding Dependency FailuresDeclaring Resource DefaultsReviewing Resource Processing

Managing DependenciesReferring to ResourcesOrdering ResourcesAssuming Implicit DependenciesTriggering Refresh EventsChaining Resources with ArrowsProcessing with CollectorsUnderstanding Puppet OrderingDebugging Dependency CyclesAvoiding the Root User TrapUtilizing StagesReviewing Resource Relationships
Replacing Deprecated FeaturesJunking the Ruby DSLUpgrading Config EnvironmentsRemoving Node InheritenceDisabling puppet kickQualifying Relative Class NamesLosing the Search FunctionReplacing ImportDocumenting Modules with Puppet StringsInstalling the Tagmail Report ProcessorQuerying PuppetDBPreparing for the UpgradeValidating Variable NamesQuoting StringsPreventing Numeric AssignmentTesting Boolean FactsQualifying Defined TypesAdding Declarative PermissionsRemoving Cron PurgeReplacing MSI Package ProviderAdjusting Networking FactsTesting with the Future ParserUsing Directory EnvironmentsDuplicating a Master or NodeEnhancing Older ManifestsAdding else to unlessCalling Functions in StringsMatching String RegexpsLetting Expressions Stand AloneChaining AssignmentsChaining Expressions with a SemicolonUsing Hash and Array LiteralsConfiguring Error Reporting
Best Practices for Writing ManifestsLearning More About Puppet Manifests
Verifying the Production EnvironmentCreating the Test EnvironmentChanging the Base Module PathSkipping Ahead
Introducing HieraSelecting a Data SourceSelecting a Hiera BackendCreating Hiera DataConfiguring HieraVersionDefaultsHierarchyMerge StrategyComplete ExampleLooking Up Hiera DataChecking Hiera Values from the Command LinePerforming Hiera Lookups in a ManifestTesting Merge StrategyProviding Global Data
Finding ModulesPuppet ForgePublic GitHub RepositoriesInternal RepositoriesEvaluating Module QualityPuppet SupportedPuppet ApprovedQuality ScoreCommunity RatingInstalling ModulesInstalling from a Puppet ForgeInstalling from GitHubTesting a Single ModuleDefining Config with HieraAssigning Modules to NodesUsing Hiera for Module AssignmentAssigning Classes to Every NodeAltering the Class List per NodeAvoiding Node Assignments in ManifestsUpgrading from Puppet 2 or 3Examining a ModuleReviewing Modules
Choosing a Module NameAvoiding Reserved NamesGenerating a Module SkeletonModifying the Default SkeletonUnderstanding Module StructureInstalling the ModuleCreating a Class ManifestWhat Is a Class?Declaring Class ResourcesAccepting InputSharing FilesTesting File SynchronizationSynchronizing DirectoriesParsing TemplatesCommon SyntaxUsing Puppet EPP TemplatesUsing Ruby ERB TemplatesCreating Readable TemplatesTesting the ModulePeeking Beneath the HoodBest Practices for Module DesignReviewing Custom Modules
Validating Input with Data TypesValid TypesValidating ValuesTesting ValuesComparing Strings with Regular ExpressionsMatching a Regular ExpressionRevising the ModuleLooking Up Input from HieraNaming Parameters Keys CorrectlyUsing Array and Hash MergesUnderstanding Lookup MergeDebugging LookupSpecifying Merge Strategy in DataReplacing Direct Hiera CallsBuilding SubclassesCreating New Resource TypesUnderstanding Variable ScopeUsing Out-of-Scope VariablesUnderstanding Top ScopeUnderstanding Node ScopeUnderstanding Parent ScopeTracking Resource Defaults ScopeAvoiding Resource Default BleedRedefining VariablesCalling Other ModulesSourcing a Common DependencyUsing a Different ModuleOrdering DependenciesDepending on Entire ClassesPlacing Dependencies Within Optional ClassesNotifying Dependencies from Dynamic ResourcesSolving Unknown Resource DependenciesContaining ClassesCreating Reusable ModulesAvoiding Fixed Values in Attribute ValuesEnsuring Fixed Values for Resource NamesDefining Defaults in a Params ManifestBest Practices for Module ImprovementsReviewing Module Improvements
Adding Custom FactsExternal FactsCustom (Ruby) FactsDebuggingUnderstanding Implementation IssuesDefining FunctionsPuppet FunctionsRuby FunctionsUsing Custom FunctionsCreating Puppet TypesDefining EnsurableAccepting Params and PropertiesValidating Input ValuesDefining Implicit DependenciesLearning More About Puppet TypesAdding New ProvidersDetermining Provider SuitabilityAssigning a Default ProviderDefining Commands for UseEnsure the Resource StateAdjusting PropertiesProviding a List of InstancesTaking Advantage of CachingLearning More About Puppet ProvidersIdentifying New FeaturesBinding Data Providers in ModulesUsing Data from HieraPerforming Lookup QueriesRequirements for Module PluginsReviewing Module Plugins
Learning MarkdownWriting a Good READMEDocumenting the Classes and TypesInstalling YARD and Puppet StringsFixing the HeadersListing ParametersDocumenting Variable ReferencesShowing ExamplesListing Authors and CopyrightDocumenting FunctionsGenerating DocumentationUpdating Module MetadataIdentifying the LicensePromoting the ProjectIndicating CompatibilityDefining RequirementsListing DependenciesIdentifying a Module Data SourceUpdating Old MetadataMaintaining the Change LogEvolving and ImprovingBest Practices for Documenting Modules
Installing DependenciesInstalling RubyAdding BeakerBundling DependenciesPreparing Your ModuleDefining FixturesDefining RSpec Unit TestsDefining the Main ClassPassing Valid ParametersFailing Invalid ParametersTesting File CreationValidating Class InclusionUsing Facts in TestsUsing Hiera InputDefining Parent Class ParametersTesting FunctionsAdding an Agent ClassTesting Other TypesCreating Acceptance TestsInstalling Ruby for System TestsDefining the NodesetConfiguring the Test EnvironmentCreating an Acceptance TestRunning Acceptance TestsUsing Skeletons with Testing FeaturesFinding DocumentationReviewing Testing Modules
Updating the Module MetadataPackaging a ModuleUploading a Module to the Puppet ForgePublishing a Module on GitHubAutomating Module PublishingGetting Approved Status from Puppet Labs
Understanding the Catalog BuilderNodeAgentServerPlanning for Puppet ServerThe Server Is Not the NodeThe Node Is Not the ServerStore Server Data Files SeparatelyFunctions Run on the ServerChoosing Puppet Master Versus Puppet ServerUpgrading Easily with Puppet MasterEmbracing the Future with Puppet ServerWhy There’s Really No ChoiceEnsuring a High-Performance Server
Starting the puppetmaster VMInstalling the Puppet MasterConfiguring a Firewall for the Puppet MasterRunning the WEBrick ServerTesting with the Puppet Master ServiceScaling the Puppet Master with PassengerInstalling ApacheInstalling Phusion PassengerConfiguring the Puppet MasterIPv6 Dual-Stack Puppet MasterDebugging Puppet Master
Starting the puppetserver VMInstalling Puppet ServerConfiguring a Firewall for Puppet ServerConfiguring Puppet ServerDefining Server PathsLimiting Memory UsageConfiguring TLS CertificatesAvoiding Obsolete SettingsConfiguring Server LogsConfiguring Server AuthenticationRunning Puppet ServerAdding Ruby GemsIPv6 Dual-Stack Puppet Server
Creating a Key PairAuthorizing the NodeDownloading the First CatalogInstalling Hiera Data and ModulesTesting with a Client NodeLearning More About Puppet Server
Migrating the Puppet Master ConfigSynchronizing All EnvironmentsCopying Hiera DataMoving the MCollective Config DirectoryRemoving Node InheritanceTesting a Client NodeUpgrading Clients
Using Server Data in Your ManifestsTrusted FactsServer FactsServer Configuration SettingsBacking Up Files Changed on NodesProcessing Puppet Node ReportsEnabling Transmission of ReportsRunning Audit InspectionsStoring Node ReportsLogging Node ReportsTransmitting Node Reports via HTTPTransmitting Node Reports to PuppetDBEmailing Node ReportsCreating a Custom Report Processor
Reviewing Node AuthenticationAutosigning Agent CertificatesName-Based AutosigningPolicy-Based AutosigningNaive AutosigningUsing an External Certificate AuthorityDistributing Certificates ManuallyInstalling Certificates on the ServerDisabling CA on a Puppet ServerDisabling CA on a Puppet MasterUsing Different CAs for Servers and AgentsDistributing the CA Revocation ListLearning More About TLS Authentication
Using a Node TerminusRunning an External Node ClassifierQuerying LDAPStarting with Community ExamplesDeploying Puppet Servers at ScaleKeeping Distinct DomainsSharing a Single Puppet CAUsing a Load BalancerManaging Geographically Dispersed ServersManaging Geographically Dispersed NodesFalling Back to Cached CatalogsMaking the Right ChoiceBest Practices for Puppet ServersReviewing Puppet Servers
Using Puppet DashboardInstalling Dashboard DependenciesEnabling Puppet DashboardViewing node statusUsing Dashboard as a Node ClassifierImplementing Dashboard in ProductionEvaluating Alternative DashboardsPuppetboardPuppet ExplorerPanoPuppetENC DashboardForemanUpgrading to the Enterprise ConsoleViewing StatusClassifying NodesInspecting EventsTracking ChangesControlling AccessEvaluating Puppet EnterpriseFinding Plugins and Tools
Creating a Windows Virtual MachineCreating a VirtualBox Windows VMAdding an Internal Network AdapterConnecting the Windows Installation MediaConfiguring the Internal Network AdapterInstalling Puppet on WindowsConfiguring Puppet on WindowsRunning Puppet InteractivelyStarting the Puppet ServiceDebugging Puppet ProblemsWriting Manifests for WindowsFinding Windows-Specific ModulesConcluding Thoughts on Puppet Windows
Understanding Environment IsolationEnabling Directory EnvironmentsAssigning Environments to NodesConfiguring an EnvironmentEnvironment Configuration FileChoosing a Manifest PathDisabling the Environment CacheUsing Environment DataRemoving Environment Paths from Global DataUtilizing Hiera Hierarchy in EnvironmentsUsing Custom Backends in EnvironmentsQuerying Environment Data from HieraStrategizing How to Use EnvironmentsPromoting Change Through LayersSolving One-Off Problems Using EnvironmentsSupporting Diverse Teams with EnvironmentsManaging Environments with r10kListing Modules in the PuppetfileCreating a Control RepositoryConfiguring r10k SourcesAdding New EnvironmentsPopulating a New InstallationUpdating a Single EnvironmentReplicating Hiera DataInvalidating the Environment CacheRestarting JRuby When Updating PluginsReviewing Environments
Configuring MCollectiveEnabling the Puppet Labs RepositoryInstalling the MCollective ModuleGenerating PasswordsConfiguring Hiera for MCollectiveEnabling the MiddlewareConnecting MCollective ServersValidating the InstallationCreating Another ClientInstalling MCollective Agents and ClientsSharing Facts with PuppetPulling the Puppet StringsViewing Node InventoryChecking Puppet StatusDisabling the Puppet AgentInvoking Ad Hoc Puppet RunsLimiting Targets with FiltersProviding a List of TargetsLimiting ConcurrencyManipulating Puppet Resource TypesComparing to Puppet Application OrchestrationLearning More About MCollective
Managing Network Devices with Puppet DeviceEnabling SSH on the SwitchConfiguring the Puppet Proxy AgentInstalling the Device_Hiera ModuleDefining Resource Defaults in HieraCentralizing VLAN ConfigurationApplying Default Configs to InterfacesCustomizing Interface ConfigurationsTesting Out the Switch ConfigurationAdding Resource Types and ProvidersMerging Defaults with Other ResourcesUsing the NetDev Standard LibraryFinding NetDev Vendor ExtensionsCreating a NetDev Device ObjectReducing Duplication with Device_HieraPuppetizing Cisco Nexus SwitchesConfiguring the Puppet ServerPreparing the NX-OS DeviceInstalling the NX-OS Puppet AgentEnabling the NX-OS Puppet AgentManaging ConfigurationPuppetizing Juniper DevicesSupported DevicesInstalling Modules on the Puppet ServerPreparing the Junos DeviceInstalling the Junos Puppet AgentCreating the Puppet UserAdjusting Physical Interface SettingsSimplifying Layer-2 VLANsEnabling Link AggregationDefining Ad Hoc Configuration ParametersDistributing Junos Event ScriptsRunning Puppet AutomaticallyTroubleshootingBest Practices for Network DevicesReviewing Network Devices
Managing ChangeExpecting ChangeControlling Rate of ChangeTracking ChangeChoosing Puppet Apply Versus Puppet ServerBenefits of Puppet ApplyBenefits of Puppet ServerBenefits SharedSummarizing the DifferencesCreating a Private Puppet ForgePulpPuppet Forge ServerDjango ForgeGood PracticesIndenting HeredocSplaying Puppet Agent Cron JobsCleaning Puppet ReportsTrimming the File BucketDrinking the Magic Monkey JuiceHating on Params.ppDisabling EnvironmentsTracking ProvidersBreaking the RulesWorking Good, Fast, CheapChoosing Fight or FlightLetting the Strings Pull YouLeveraging Puppet for Small ChangesTossing Declarative to the WindAllowing Anyone to sudo puppet
Accessing Community SupportEngaging Puppet Labs SupportContacting the Author
Some Best Practices May Not Work for YouLearning to Fail is the Secret to Success
Debian and UbuntuFedoraOther Platforms
IP TablesUncomplicated Firewall
Ruby for MacRuby for WindowsRuby for Linux

Content preview from Learning Puppet 4

Chapter 30. Controlling Puppet with MCollective

In this chapter, we will configure and enable the orchestration tool that shipped as part of Puppet 4: the Marionette Collective, or MCollective. You’ll learn how to use MCollective to control the Puppet agent on your nodes.

MCollective can:

Query, start, stop, and restart the Puppet agent.
Run the Puppet agent with special command-line options.
Query and make changes to the node using Puppet resources.
Choose nodes to act on based on the Puppet classes or facts on the node.
Control concurrency of Puppet runs across a group of nodes.

Tip

If you have used puppet kick in the past, you are likely aware that Puppet Labs has obsoleted and has removed support for it in Puppet 4. The MCollective Puppet agent replaces Puppet kick in both the community and Puppet Enterprise product lines, and provides significantly more features and functionality.

Configuring MCollective

In this section, you will:

Install ActiveMQ to provide middleware on the Puppet server.
Install the MCollective server on every node.
Install the MCollective clients on a system of your choice.
Enable TLS encryption for all connections.
Install the Puppet agent plugin for ad hoc control of Puppet.

Enabling the Puppet Labs Repository

Puppet Labs provides Yum and Apt repositories containing packages for the MCollective server, clients, and some officially supported plugins. This community repository supplements the OS vendor repositories for the more popular Linux distributions. ...