book

Learning Chaos Engineering

by Russ Miles

July 2019

Beginner

175 pages

3h 55m

English

O'Reilly Media, Inc.

Read now

Unlock full access

AudienceWhat This Book Is AboutWhat This Book Is Not AboutAbout the SamplesConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Chaos Engineering DefinedChaos Engineering Addresses the Whole Sociotechnical SystemLocations of Dark DebtThe Process of Chaos EngineeringThe Practices of Chaos EngineeringSandbox/Staging or Production?Chaos Engineering and ObservabilityIs There a “Chaos Engineer”?Summary
Start with Experiments?Gathering HypothesesIncident AnalysisSketching Your SystemCapturing “What Could Possibly Go Wrong?”Introducing Likelihood and ImpactBuilding a Likelihood-Impact MapAdding What You Care AboutCreating Your Hypothesis BacklogSummary
What Is a Game Day?Planning Your Game DayPick a HypothesisPick a Style of Game DayDecide Who Participates and Who ObservesDecide WhereDecide When and For How LongDescribe Your Game Day ExperimentGet Approval!Running the Game DayConsider a “Safety Monitor”Summary
Installing Python 3Installing the Chaos Toolkit CLISummary
Setting Up the Sample Target SystemA Quick Tour of the Sample SystemExploring and Discovering Evidence of WeaknessesRunning Your ExperimentUnder the Skin of chaos runSteady-State Deviation Might Indicate “Opportunity for Improvement”Improving the SystemValidating the ImprovementSummary
The Target SystemThe Platform: A Three-Node Kubernetes ClusterThe Application: A Single Service, Replicated Three TimesThe People: Application Team and Cluster AdministratorsHunting for a WeaknessNaming Your ExperimentDefining Your Steady-State HypothesisInjecting Turbulent Conditions in an Experiment’s MethodUsing the Kubernetes Driver from Your MethodBeing a Good Citizen with RollbacksBringing It All Together and Running Your ExperimentOvercoming a Weakness: Applying a Disruption BudgetSummary
Sharing Experiment DefinitionsMoving Values into ConfigurationSpecifying Configuration Properties as Environment VariablesExternalizing SecretsScoping SecretsSpecifying a Contribution ModelCreating and Sharing Human-Readable Chaos Experiment ReportsCreating a Single-Experiment Execution ReportCreating and Sharing a Multiple Experiment Execution ReportSummary

Creating Your Own Custom Driver with No Custom CodeImplementing Probes and Actions with HTTP CallsImplementing Probes and Actions Through Process CallsCreating Your Own Custom Chaos Driver in PythonCreating a New Python Module for Your Chaos Toolkit Extension ProjectAdding the ProbeSummary
Experiment “Controls”Enabling ControlsEnabling a Control Inline in an ExperimentEnabling a Control GloballySummary
Adding Logging to Your Chaos ExperimentsCentralized Chaos Logging in ActionTracing Your Chaos ExperimentsIntroducing OpenTracingApplying the OpenTracing ControlSummary
Creating a New Chaos Toolkit Extension for Your ControlsAdding Your (Very) Simple Human Interaction ControlSkipping or Executing an Experiment’s ActivitySummary
What Is Continuous Chaos?Scheduling Continuous Chaos Using cronCreating a Script to Execute Your Chaos TestsAdding Your Chaos Tests Script to cronScheduling Continuous Chaos with JenkinsGrabbing a Copy of JenkinsAdding Your Chaos Tests to a Jenkins BuildScheduling Your Chaos Tests in Jenkins with Build TriggersSummary
The Default Chaos CommandsDiscovering What’s Possible with the chaos discover CommandAuthoring a New Experiment with the chaos init CommandChecking Your Experiment with the chaos validate CommandExtending the Chaos Commands with Plug-ins