book

Splunk: Enterprise Operational Intelligence Delivered

by Betsy Page Sigman, Erickson Delgado, Josh Diakun, Paul R Johnson, Derek Mock, Ashish Kumar Tulsiram Yadav

February 2017

Beginner to intermediate

962 pages

21h 26m

English

Packt Publishing

Read now

Unlock full access

What this learning path covers
Downloading the example codeErrataPiracyQuestions

Your Splunk.com accountObtaining a Splunk.com account
Logging in the first timeRun a simple search
Installing an add-on
Splunk and big dataStreaming dataLatency of dataSparseness of data
Machine dataWeb logsData filesSocial media dataOther data types
Anatomy of a searchSearch pipeline
Creating a data modelAdding attributes to objectsCreating child objectsCreating an attribute based on a regular expression
The Pivot EditorCreating a chart from a PivotCreating an area chartCreating a pie chart showing destination details by airport codeSingle value with trending sparkline
Data classification with event types
Creating effective dashboards
Gathering information and business requirementsDynamic form-based dashboardCreating a Status Distribution panelCreating the Status Types Over Time panelCreating the Hits vs Response Time panelArranging the dashboardPanel optionsPie chart - status distributionStacked area chart - Status Types Over TimeColumn with line overlay combo chart - Hits vs Response Time
Single Value Panels with color rangesCreating panels by cloningSingle Value Panels with trendsReal-time column charts with line overlays
Introduction to Splunk SDKs
PrerequisitesCreating a CRON JobCreating a saved search
HTTP serverRendering the chart
What is the HEC?
Logging in dataUsing a token with dataSending out the data requestVerifying the tokenIndexing the dataEnabling the HECGenerating an HEC authentication tokenHow to test the HEC with cURL and PowerShellUsing the HEC with dynamic UI eventsJavaScript logging with the HEC
Temporary indexes and oneshot indexing
SubsearchUsing appendUsing joinUsing eval and ifUsing eval and match with a case function
Including clear key-value pairsCreating events that are understandable to human readersRemember to use timestamps for all eventsBe sure your identifiers are uniqueLog using text format, not binaryUse formats that developers can use easilyLog what you think might be useful at some pointCreate use categories with meaningInclude the source of the log eventMinimize the number of multi-line events
Introduction
Getting readyHow to do it…How it works…There's more…Adding a file or directory data input via the CLIAdding a file or directory input via inputs.confOne-time indexing of data files via the Splunk CLIIndexing the Windows event logsSee also
Getting readyHow to do it…How it works…There's more…Adding a network input via the CLIAdding a network input via inputs.confSee also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…There's more…See also
Getting readyHow to do it…How it works…There's more…Add the receiving indexer via outputs.conf
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…There's more…Adding event types and tags via eventtypes.conf and tags.confSee also
Introduction
Getting readyHow to do it…How it works…There's more…Tabulating every fieldRemoving fields, then tabulating everything else
Getting readyHow to do it…How it works…There's more…Searching for the top 10 accessed web pagesSearching for the most accessed pages by userSee also
Getting readyHow to do it…How it works…There's more…Searching for the web browser data for the most used OS typesSee also
Getting readyHow to do it…How it works…There's more…Searching for the top 10 using stats instead of topSee also
Getting readyHow to do it…How it works…There's more…Totaling success and error web page response codesSee also
Getting readyHow to do it…How it works…There's more…Displaying web page response time by actionSee also
Getting readyHow to do it…How it works…There's more…Searching for the percentage of cart additions from product viewsSee also
Getting readyHow to do it…How it works…There's more…See also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Introduction
Getting readyHow to do it…How it works…There's more…Changing dashboard permissions
Getting readyHow to do it…How it works…There's more…Searching for the top 10 accessed web pagesSee also
Getting readyHow to do it…How it works…There's more…Coloring the value based on rangesAdding trends and sparklines to the valuesSee also
Getting readyHow to do it…How it works…There's more…See also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…There's more…Method requests, views, and response times by hostSee also
Getting readyHow to do it…How it works…There's more…Using time series data points with a scatter chartSee also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Introduction
Getting readyHow to do it…How it works…There's more…Creating an application from another applicationDownloading and installing a Splunk appSee also
Getting readyHow to do it…How it works…There's more…Changing permissions of saved reportsSee also
Getting readyHow to do it…How it works…There's more…Modifying the Simple XML directlySee also
Getting readyHow to do it…How it works…There's more…Disabling the drilldown feature in tables and chartsSee also
Getting readyHow to do it…How it works…There's more…Adding a Submit button to your formSee also
Getting readyHow to do it…How it works…There's more…Adding an overlay to the Sessions Over Time chartSee also
Getting readyHow to do it…How it works…There's more…Adding a map panel using Simple XMLMapping different distributions by areaSee also
Getting readyHow to do it…How it works…See also
Introduction
Getting readyHow to do it…How it works…There's more…Searching data models using the search interfaceSee also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…There's more…Viewing data model and acceleration summary informationAdvanced configuration of data model accelerationSee also
Getting readyHow to do it…How it works…There's more…Pivot searching using the pivot command and search interfaceSee also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Introduction
Getting readyHow to do it…How it works…There's more…Starts with a website visit, ends with a checkoutDefining maximum pause, span, and events in a transactionSee also
Getting readyHow to do it…How it works…There's more…Calculating the average execution time without using a joinSee also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…How it works…There's more…Analyzing relationships of DB actions to memory utilizationSee also
Getting readyHow to do it…How it works…There's more…Predicting the total number of items purchasedPredicting the average response time of function callsSee also
Getting readyHow to do it…How it works…There's more…The anomalies commandThe anomalousvalues commandThe anomalydetection commandThe cluster commandSee also
Getting readyHow to do it…How it works…There's more…Creating logic for urgencySee also
Introduction
Getting readyHow to do it…How it works…There's more…Manually adding the lookup to SplunkSee also
Getting readyHow to do it…How it works…There's more…Modifying an existing saved search to populate a lookup tableSee also
Getting readyHow to do it…How it works…There's more…Use the Splunk KV store to maintain the session state tableSee also
Getting readyHow to do it…How it works…There's more…Enabling automatic external field lookupsSee also
Getting readyHow to do it…How it works…There's more…Limiting workflow actions by event typesSee also
Getting readyHow to do it…How it works…There's more…Triggering a Google search from the chart drilldown optionsSee also
Getting readyHow to do it…How it works…There's more…Adding a workflow action manually in SplunkSee also
Getting readyHow to do it…How it works…There's more…Use DB Connect for direct external DB lookupsSee also
Introduction
Getting readyHow to do it…How it works…There's more…Viewing triggered alerts in Splunk's Alert managerSee also
Getting readyHow to do it…How it works…There's more…Building alerts via a configuration fileEditing alert configuration attributes using Advanced editIdentify the real-time searches that are runningSee also
Getting readyHow to do it…How it works…There's more…Alerting on abnormal user purchases without checkoutsSee also
Getting readyHow to do it…How it works…There's more…See also
Getting readyHow to do it…How it works…There's more…Adding an RSS feed notification action to an alertSee also
Introduction
Getting readyHow to do it…How it works…There's more…Generating the summary more frequentlyAvoiding summary index overlaps and gapsSee also
Getting readyHow to do it…How it works…There's more…Backfilling a summary index from within a search directlySee also
Getting readyHow to do it…How it works…There's more…Viewing the status of an accelerated reportSee also
Introduction
Getting readyHow to do it...How it works...There's more…
Getting readyHow to do it...How it works...There's more…Changing the time range on the search managerSee also
Getting readyHow to do it...How it works...See also
Getting readyHow to do it...How it works...There's more…See also
Getting readyHow to do it...How it works...There's more…Authenticating with a session tokenSee also
Getting readyHow to do it...How it works...There's more...Paginating the results of your searchSee also
Getting readyHow to do it...How it works...See also
Getting readyHow to do it...How it works...See also
Splunk's architectureThe need for parallelizationIndex parallelization
Pipeline parallelizationThe search schedulerSummary parallelization
System requirementsUses of the key-value storeComponents of the key-value storeManaging key-value store collections via RESTExamplesReplication of the key-value store
Enabling HTTPS for Splunk WebEnabling HTTPS for the Splunk forwarderSecuring a password with SplunkThe access control list
Splunk apps and technology add-onsWhat is a Splunk app?What is a technology add-on?
Creating the Splunk application and technology add-onPackaging the applicationInstalling a Splunk app via Splunk WebInstalling the Splunk app manually
Building an add-onInstalling a technology add-on
Deep diving into various input methods and sourcesData sourcesStructured dataWeb and cloud servicesIT operations and network securityDatabasesApplication and operating system dataData input methodsFiles and directoriesNetwork sourcesWindows data
HTTP Event Collector and configurationHTTP Event CollectorConfiguration via Splunk WebManaging the Event Collector tokenThe JSON API formatAuthenticationMetadataEvent data
Event configurationCharacter encodingEvent line breakingTimestamp configurationHost configurationConfiguring a static host value – files and directoriesConfiguring a dynamic host value – files and directoriesConfiguring a host value – events
Data and indexesAccessing dataThe index commandThe eventcount commandThe datamodel commandThe dbinspect commandThe crawl commandManaging dataThe input commandThe delete commandThe clean commandSummary indexing
The search commandThe sendmail commandThe localop command
The append commandThe appendcols commandThe appendpipe commandThe join command
The reltime commandThe localize command
The eval commandThe xmlkv commandThe spath commandThe makemv commandThe fillnull commandThe filldown commandThe replace command
The fields commandThe searchtxn commandThe head / tail commandThe inputcsv commandThe outputcsv command
ReportsThe makecontinuous commandThe addtotals commandThe xyseries command
The iplocation commandThe geostats command
The anomalies commandThe anomalousvalue commandThe cluster commandThe kmeans commandThe outlier commandThe rare command
The predict commandThe trendline commandThe x11 command
The correlate commandThe associate commandThe diff commandThe contingency command
Prerequisites – configuration settings
Tables – Data overlayTables – SparklineSparkline – Filling and changing colorSparkline – The max value indicatorSparkline – A bar styleTables – An icon set
Charts – ColoringChart overlayBubble charts
Dynamic drilldownThe x-axis or y-axis value as a token to a formDynamic drilldown to pass a respective row's specific column valueDynamic drilldown to pass a fieldname of a clicked valueContextual drilldownThe URL field value drilldownSingle value drilldown
Sunburst sequenceWhat is a sunburst sequence?ExampleImplementation
ExampleSyntaxSearch queryImplementation
ExampleSearch queryImplementation
ExampleSearch queryImplementation
ExampleImplementation
ExampleSearch queryImplementation
ExampleImplementation
ExampleImplementation
ExampleWhat is the use of such custom decorations?Implementation
Dashboard controlsHTML dashboardDisplay controlsExample and implementationSyntaxForm input controlsExample and implementationPanel controlsExample and implementationEnabling/disabling refresh timeDisabling the manual refresh linkEnabling auto refresh
ExampleImplementation
Eval tokensSyntax of the eval tokenExampleImplementationCustom tokensExampleImplementation
ExampleImplementation
Link switcherExample and implementationButton switcherExample and implementation
Layout customizationPanel widthExampleImplementationGroupingExampleSingle-value groupingVisualization groupingImplementationPanel toggleExampleImplementationImage overlayExampleWhat is the use of image overlay?Where can image overlay be used?Implementation
Example and implementation
What is alerting?AlertingThe featuresImplementationExample
Index replicationStandalone environmentDistributed environmentReplicationSearchingFailures
ExampleImplementation
Accessing field extractorUsing field extractorExampleRegular expressionDelimiter
Need for data accelerationData model acceleration
Time rangeSearch modesScope of searchingSearch terms
splunkd logSearch log
The Splunk SDK
Importing the Splunk API in PythonConnecting and authenticating the Splunk serverSplunk APIsCreating and deleting an indexCreating inputUploading filesSaved searchesSplunk searches
The setupUsing R with Splunk
The setupUsing Tableau with Splunk
Storage optimization

Content preview from Splunk: Enterprise Operational Intelligence Delivered

Splunk buckets

The Splunk Enterprise %stores its index's data into buckets organized by age. Basically, it is a directory containing events of a specific period. There can be several buckets at the same time in the various stages of the bucket life cycle.

A bucket moves from one stage to another depending upon its age, size, and so on, as per the defined conditions. The Splunk bucket stages are Hot, Warm, Cold, Frozen, and Thawed. Splunk buckets play a very important role in the performance of search results and hence they should be properly configured as per the requirements.

The following image shows the life cycle of Splunk buckets:

Let us understand ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Splunk Operational Intelligence Cookbook

Big Data Analytics Using Splunk: Deriving Operational Intelligence from Social Media, Machine Data, Existing Data Warehouses, and Other Real-Time Streaming Sources

Peter Zadrozny, Raghu Kodali

Publisher Resources

ISBN: 9781787288256

Splunk: Enterprise Operational Intelligence Delivered

by Betsy Page Sigman, Erickson Delgado, Josh Diakun, Paul R Johnson, Derek Mock, Ashish Kumar Tulsiram Yadav

Splunk buckets

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

You might also like

Splunk Operational Intelligence Cookbook

Splunk Operational Intelligence Cookbook - Second Edition

IBM Tivoli Storage Manager as a Data Protection Solution

Big Data Analytics Using Splunk: Deriving Operational Intelligence from Social Media, Machine Data, Existing Data Warehouses, and Other Real-Time Streaming Sources

Publisher Resources