book

Practical Monitoring

by Mike Julian

October 2017

Beginner to intermediate

170 pages

3h 58m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Who Should Read This BookWhy I Wrote This BookA Word on Monitoring TodayNavigating This BookOnline ResourcesConventions Used in This BookUsing Code ExamplesO’Reilly SafariHow to Contact UsAcknowledgments
Anti-Pattern #1: Tool ObsessionMonitoring Is Multiple Complex Problems Under One NameAvoid Cargo-Culting ToolsSometimes, You Really Do Have to Build ItThe Single Pane of Glass Is a MythAnti-Pattern #2: Monitoring-as-a-JobAnti-Pattern #3: Checkbox MonitoringWhat Does “Working” Actually Mean? Monitor That.OS Metrics Aren’t Very Useful—for AlertingCollect Your Metrics More OftenAnti-Pattern #4: Using Monitoring as a CrutchAnti-Pattern #5: Manual ConfigurationWrap-Up
Pattern #1: Composable MonitoringThe Components of a Monitoring ServicePattern #2: Monitor from the User PerspectivePattern #3: Buy, Not BuildIt’s CheaperYou’re (Probably) Not an Expert at Architecting These ToolsSaaS Allows You to Focus on the Company’s ProductNo, Really, SaaS Is Actually BetterPattern #4: Continual ImprovementWrap-Up
What Makes a Good Alert?Stop Using Email for AlertsWrite RunbooksArbitrary Static Thresholds Aren’t the Only WayDelete and Tune AlertsUse Maintenance PeriodsAttempt Automated Self-Healing FirstOn-CallFixing False AlarmsCutting Down on Needless FirefightingBuilding a Better On-Call RotationIncident ManagementPostmortemsWrap-Up
Before Statistics in Systems OperationsMath to the Rescue!Statistics Isn’t MagicMean and AverageMedianSeasonalityQuantilesStandard DeviationWrap-Up
Business KPIsTwo Real-World ExamplesYelpRedditTying Business KPIs to Technical MetricsMy App Doesn’t Have Those Metrics!Finding Your Company’s Business KPIsWrap-Up
The Cost of a Slow AppTwo Approaches to Frontend MonitoringDocument Object Model (DOM)Frontend Performance MetricsOK, That’s Great, but How Do I Use This?LoggingSynthetic MonitoringWrap-Up
Instrumenting Your Apps with MetricsHow It Works Under the HoodMonitoring Build and Release PipelinesHealth Endpoint PatternApplication LoggingWait a Minute…Should I Have a Metric or a Log Entry?What Should I Be Logging?Write to Disk or Write to Network?Serverless / Function-as-a-ServiceMonitoring Microservice ArchitecturesWrap-Up

Standard OS MetricsCPUMemoryNetworkDiskLoadSSL CertificatesSNMPWeb ServersDatabase ServersLoad BalancersMessage QueuesCachingDNSNTPMiscellaneous Corporate InfrastructureDHCPSMTPMonitoring Scheduled JobsLoggingCollectionStorageAnalysisWrap-Up
The Pains of SNMPWhat Is SNMP?How Does It Work?A Word on SecurityHow Do I Use SNMP?Interface MetricsInterface and LoggingRecapConfiguration TrackingVoice and VideoRoutingSpanning Tree Protocol (STP)ChassisCPU and MemoryHardwareFlow MonitoringCapacity PlanningWorking BackwardForecastingWrap-up
Monitoring and ComplianceUser, Command, and Filesystem AuditingSetting Up auditdauditd and Remote LogsHost Intrusion Detection System (HIDS)rkhunterNetwork Intrusion Detection System (NIDS)Wrap-Up
Business KPIsFrontend MonitoringApplication and Server MonitoringSecurity MonitoringAlertingWrap-Up
Demo AppMetadataEscalation ProcedureExternal DependenciesInternal DependenciesTech StackMetrics and LogsAlerts

Content preview from Practical Monitoring

Chapter 11. Conducting a Monitoring Assessment

We’ve reached the final chapter of the book, folks. You have, I hope, learned many new things. This last chapter will take you through a fictionalized example of applying all the lessons of this book at once using an exercise I do with my own consulting clients: a monitoring assessment.

Performing a monitoring assessment on your environment is a great way to systematically determine what you should be monitoring and why. The end result is a clearer understanding of the behavior of your app and underlying infrastructure. It’s by no means exhaustive or perfect, but rather, it’s intended to be a starting point to get you thinking about what matters and what doesn’t.

Business KPIs

To start off, we need to figure out exactly what Tater.ly does. After a chat with the CEO, we’ve learned the following:

Tater.ly’s mission is to help french-fry aficionados find the best french fries in all the land. Users come to Tater.ly to look up restaurants and read reviews about their french fries, as well as post their own reviews. The french fries are also rated on a scale of one to five, with five being the best. Restaurants can create their own pages or users can create them. Restaurants can “claim” their pages if the page already exists. Tater.ly makes money through advertising by placing a Featured Fry at the top of search results, with restaurants paying an advertising fee for the slot. The ad fees are based on number of impressions—that is, the ...