book

Monitoring with Graphite

by Jason Dixon

March 2017

Intermediate to advanced

290 pages

7h 30m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who Should Read This BookWhy I Wrote This BookA Word on Monitoring TodayNavigating This BookConventions Used in This BookO’Reilly SafariHow to Contact UsAcknowledgments
What Is Time-Series Data?Time-Series DatabasesStorage ConsiderationsPrioritizing WorkloadsWhat Is the History of Graphite?What Makes Graphite Unique?Simple Metrics FormatGraphing APIRapid PrototypingRich Statistical LibraryChained FunctionsCase Studies: Who Uses Graphite in Production?Booking.comGitHubEtsyElectronic ArtsWhy Should I Use Graphite?
Three Tenets of MonitoringFault DetectionAlertingCapacity PlanningRethinking the Poll/Pull ModelPull ModelPush ModelWhere Does Graphite Fit into the Picture?Composable Monitoring SystemsTelemetryMetrics RouterAggregationState EngineNotification RoutersStorage EngineVisualizationConclusion
Carboncarbon-cachecarbon-relaycarbon-aggregatorFiltering MetricsInternal StatisticsNetwork Security ConsiderationsWhisperHow Do Whisper Files Get Created?Retention Policies and ArchivesThe Laws of Whisper ArchivesCalculating Whisper File SizesDeconstructing a Whisper FileWhich Archive Handles My Query?Aggregation MethodsxFilesFactorPlanning Your NamespacesPerformance ConsiderationsGraphite-WebDjango FrameworkWeb ServerDatabaseMemcachedEventsStorage BackendsPutting It All TogetherBasic SetupVertical ScalingHorizontal ScalingMultisite ReplicationA Final Thought
Quick Start with SynthesizeWhere Does Graphite Store All My Files?Are Packages Available for My Distro?What Installation Methods Are Available?Should I Use Virtualenv?Using sudo EffectivelyDependenciesInstalling from SourcePreparing Your Web DatabaseConfiguring Carboncarbon.confstorage-schemas.confstorage-aggregation.confSome Final PreparationsStarting Your Carbon DaemonsConfiguring Graphite-Weblocal_settings.pySetting Up ApacheVerifying Your Graphite InstallationCarbon StatisticsFeeding New Data to CarbonBuilding Your First Graph
Finding MetricsNavigating the TreeUsing the Search FeatureWorking Smarter with the Auto-CompleterWildcardsThe Graphite Composer WindowThe Embedded ChartThe ToolbarSelecting Recent DataRefreshing the GraphSelecting a Date RangeExporting a Short URLLoading a Graph from URLSaving to My GraphsDeleting from My GraphsThe Graph Options MenuAdding a Graph TitleOverriding the Graph LegendToggling Axes and the GridApplying a Graph TemplateLine Chart ModesArea and Stacked GraphsTweaking the Y-AxisThe Graph Data DialogWhat Are Targets Anyway?Building a Carbon Performance GraphSharing Your Work
Working with FunctionsStarting with the BasicsMath and Statistical TransformsFiltering and SortingGrouping on WildcardsData Smoothing and ForecastingAdjusting Metric LabelsAlternate Output Formats
Why Do I Need a Dashboard?Graphite DashboardThird-Party DashboardsGrafanaTasseoDuskDo-It-YourselfDashingRickshaw and D3.jsConclusion
First, the BasicsThe Troubleshooting ToolbeltGenerating Metrics and BenchmarkingCPU UtilizationDisk Performance (I/O)NetworkingInspecting MetricsConfiguration SettingsCarbonGraphite-WebLoggingCarbonGraphite-WebKernel messagesFailure ScenariosThe Full DiskCPU SaturationRendering ProblemsTaking It to the Next Level
What Makes It “Hard” to Scale Graphite?Peter’s Graphite StoryThe BeginningThe Pains of PopularityClearing the Next HurdleTry, Try AgainMaximizing ResourcesAvoiding OutagesShared Web DatabaseScaling in Both DirectionsSome Final ThoughtsSummary

CarbonCommon to All Carbon typescarbon-cachecarbon-relaycarbon-aggregatorGraphite-Web

Content preview from Monitoring with Graphite

Chapter 4. Building Your First Graphite Server

I hope I haven’t scared you away with all of the mental preparation and cautionary mumbo jumbo. Graphite really is a fun and effective tool. But like any complex piece of machinery (or software), it takes an experienced operator to configure and maintain it for running at peak efficiency. The previous chapters should have imbued you with enough practical knowledge around the Graphite ecosystem to help you clear most hurdles head-on.

This chapter (finally) explains the recommended procedure for installing and configuring your own Linux instance running Graphite, Carbon, Whisper, and the database where we’ll save our graphs and dashboards later. We’ll also cover a variety of housekeeping tasks, including web access controls (authentication and authorization), fixing permissions for the storage and logging directories, and, finally, testing that everything works as expected.

Note

Note that you should be able to run all of the Graphite components on any modern UNIX or UNIX-like system, including Linux, Solaris, and the family of BSD operating systems. We only cover Linux since it is the clear leader in terms of numbers of Graphite installations, and to keep this chapter from becoming even more unwieldy than it already is.

If you’re the kind of person who prefers to learn by diving in with both feet, I have good news for you. The Synthesize project was designed to help users quickly and painlessly launch a Graphite instance. We’ll get ...