book

Splunk Operational Intelligence Cookbook - Second Edition

Name: Splunk Operational Intelligence Cookbook - Second Edition
ISBN: 9781785284991

by Josh Diakun, Jose E. Hernandez, Derek Mock, Paul R. Johnson

June 2016

Beginner to intermediate

436 pages

10h 1m

English

Packt Publishing

Read now

Unlock full access

Splunk Operational Intelligence Cookbook Second Edition
Table of Contents
Splunk Operational Intelligence Cookbook Second Edition
Credits
About the Authors
About the Reviewer
www.PacktPub.com
eBooks, discount offers, and moreWhy subscribe?Instant updates on new Packt books
Preface
What this book covers
What you need for this book
Who this book is for

Sections
Getting readyHow to do it…How it works…There's more…See also
Conventions
Reader feedback
Customer support
Downloading the example codeDownloading the color images of this bookErrataPiracyQuestions
1. Play Time – Getting Data In
Introduction
Indexing files and directories
Getting readyHow to do it…How it works…There's more…Adding a file or directory data input via the CLIAdding a file or directory input via inputs.confOne-time indexing of data files via the Splunk CLIIndexing the Windows event logsSee also
Getting data through network ports
Getting readyHow to do it…How it works…There's more…Adding a network input via the CLIAdding a network input via inputs.confSee also
Using scripted inputs
Getting readyHow to do it…How it works…See also
Using modular inputs
Getting readyHow to do it…How it works…There's more…See also
Using the Universal Forwarder to gather data
Getting readyHow to do it…How it works…There's more…Add the receiving indexer via outputs.conf
Loading the sample data for this book
Getting readyHow to do it…How it works…See also
Defining field extractions
Getting readyHow to do it…How it works…See also
Defining event types and tags
Getting readyHow to do it…How it works…There's more…Adding event types and tags via eventtypes.conf and tags.confSee also
2. Diving into Data – Search and Report
Introduction
Making raw event data readable
Getting readyHow to do it…How it works…There's more…Tabulating every fieldRemoving fields, then tabulating everything else
Finding the most accessed web pages
Getting readyHow to do it…How it works…There's more…Searching for the top 10 accessed web pagesSearching for the most accessed pages by userSee also
Finding the most used web browsers
Getting readyHow to do it…How it works…There's more…Searching for the web browser data for the most used OS typesSee also
Identifying the top-referring websites
Getting readyHow to do it…How it works…There's more…Searching for the top 10 using stats instead of topSee also
Charting web page response codes
Getting readyHow to do it…How it works…There's more…Totaling success and error web page response codesSee also
Displaying web page response time statistics
Getting readyHow to do it…How it works…There's more…Displaying web page response time by actionSee also
Listing the top viewed products
Getting readyHow to do it…How it works…There's more…Searching for the percentage of cart additions from product viewsSee also
Charting the application's functional performance
Getting readyHow to do it…How it works…There's more…See also
Charting the application's memory usage
Getting readyHow to do it…How it works…See also
Counting the total number of database connections
Getting readyHow to do it…How it works…See also
3. Dashboards and Visualizations – Making Data Shine
Introduction
Creating an Operational Intelligence dashboard
Getting readyHow to do it…How it works…There's more…Changing dashboard permissions
Using a pie chart to show the most accessed web pages
Getting readyHow to do it…How it works…There's more…Searching for the top 10 accessed web pagesSee also
Displaying the unique number of visitors
Getting readyHow to do it…How it works…There's more…Coloring the value based on rangesAdding trends and sparklines to the valuesSee also
Using a gauge to display the number of errors
Getting readyHow to do it…How it works…There's more…See also
Charting the number of method requests by type and host
Getting readyHow to do it…How it works…See also
Creating a timechart of method requests, views, and response times
Getting readyHow to do it…How it works…There's more…Method requests, views, and response times by hostSee also
Using a scatter chart to identify discrete requests by size and response time
Getting readyHow to do it…How it works…There's more…Using time series data points with a scatter chartSee also
Creating an area chart of the application's functional statistics
Getting readyHow to do it…How it works…See also
Using a bar chart to show the average amount spent by category
Getting readyHow to do it…How it works…See also
Creating a line chart of item views and purchases over time
Getting readyHow to do it…How it works…See also
4. Building an Operational Intelligence Application
Introduction
Creating an Operational Intelligence application
Getting readyHow to do it…How it works…There's more…Creating an application from another applicationDownloading and installing a Splunk appSee also
Adding dashboards and reports
Getting readyHow to do it…How it works…There's more…Changing permissions of saved reportsSee also
Organizing the dashboards more efficiently
Getting readyHow to do it…How it works…There's more…Modifying the Simple XML directlySee also
Dynamically drilling down on activity reports
Getting readyHow to do it…How it works…There's more…Disabling the drilldown feature in tables and chartsSee also
Creating a form for searching web activity
Getting readyHow to do it…How it works…There's more…Adding a Submit button to your formSee also
Linking web page activity reports to the form
Getting readyHow to do it…How it works…There's more…Adding an overlay to the Sessions Over Time chartSee also
Displaying a geographical map of visitors
Getting readyHow to do it…How it works…There's more…Adding a map panel using Simple XMLMapping different distributions by areaSee also
Scheduling PDF delivery of a dashboard
Getting readyHow to do it…How it works…See also
5. Extending Intelligence – Data Models and Pivoting
Introduction
Creating a data model for web access logs
Getting readyHow to do it…How it works…There's more…Searching data models using the search interfaceSee also
Creating a data model for application logs
Getting readyHow to do it…How it works…See also
Accelerating data models
Getting readyHow to do it…How it works…There's more…Viewing data model and acceleration summary informationAdvanced configuration of data model accelerationSee also
Pivoting total sales transactions
Getting readyHow to do it…How it works…There's more…Pivot searching using the pivot command and search interfaceSee also
Pivoting purchases by geographic location
Getting readyHow to do it…How it works…See also
Pivoting slowest responding web pages
Getting readyHow to do it…How it works…See also
Pivot charting top error codes
Getting readyHow to do it…How it works…See also
6. Diving Deeper – Advanced Searching
Introduction
Calculating the average session time on a website
Getting readyHow to do it…How it works…There's more…Starts with a website visit, ends with a checkoutDefining maximum pause, span, and events in a transactionSee also
Calculating the average execution time for multi-tier web requests
Getting readyHow to do it…How it works…There's more…Calculating the average execution time without using a joinSee also
Displaying the maximum concurrent checkouts
Getting readyHow to do it…How it works…See also
Analyzing the relationship of web requests
Getting readyHow to do it…How it works…There's more…Analyzing relationships of DB actions to memory utilizationSee also
Predicting website traffic volumes
Getting readyHow to do it…How it works…There's more…Predicting the total number of items purchasedPredicting the average response time of function callsSee also
Finding abnormally-sized web requests
Getting readyHow to do it…How it works…There's more…The anomalies commandThe anomalousvalues commandThe anomalydetection commandThe cluster commandSee also
Identifying potential session spoofing
Getting readyHow to do it…How it works…There's more…Creating logic for urgencySee also
7. Enriching Data – Lookups and Workflows
Introduction
Looking up product code descriptions
Getting readyHow to do it…How it works…There's more…Manually adding the lookup to SplunkSee also
Flagging suspect IP addresses
Getting readyHow to do it…How it works…There's more…Modifying an existing saved search to populate a lookup tableSee also
Creating a session state table
Getting readyHow to do it…How it works…There's more…Use the Splunk KV store to maintain the session state tableSee also
Adding hostnames to IP addresses
Getting readyHow to do it…How it works…There's more…Enabling automatic external field lookupsSee also
Searching ARIN for a given IP address
Getting readyHow to do it…How it works…There's more…Limiting workflow actions by event typesSee also
Triggering a Google search for a given error
Getting readyHow to do it…How it works…There's more…Triggering a Google search from the chart drilldown optionsSee also
Creating a ticket for application errors
Getting readyHow to do it…How it works…There's more…Adding a workflow action manually in SplunkSee also
Looking up inventory from an external database
Getting readyHow to do it…How it works…There's more…Use DB Connect for direct external DB lookupsSee also
8. Being Proactive – Creating Alerts
Introduction
Alerting on abnormal web page response times
Getting readyHow to do it…How it works…There's more…Viewing triggered alerts in Splunk's Alert managerSee also
Alerting on errors during checkout in real time
Getting readyHow to do it…How it works…There's more…Building alerts via a configuration fileEditing alert configuration attributes using Advanced editIdentify the real-time searches that are runningSee also
Alerting on abnormal user behavior
Getting readyHow to do it…How it works…There's more…Alerting on abnormal user purchases without checkoutsSee also
Alerting on failure and triggering a scripted response
Getting readyHow to do it…How it works…There's more…See also
Alerting when predicted sales exceed inventory
Getting readyHow to do it…How it works…There's more…Adding an RSS feed notification action to an alertSee also
9. Speeding Up Intelligence – Data Summarization
Introduction
Calculating an hourly count of sessions versus completed transactions
Getting readyHow to do it…How it works…There's more…Generating the summary more frequentlyAvoiding summary index overlaps and gapsSee also
Backfilling the number of purchases by city
Getting readyHow to do it…How it works…There's more…Backfilling a summary index from within a search directlySee also
Displaying the maximum number of concurrent sessions over time
Getting readyHow to do it…How it works…There's more…Viewing the status of an accelerated reportSee also
10. Above and Beyond – Customization, Web Framework, REST API, HTTP Event Collector, and SDKs
Introduction
Customizing the application navigation
Getting readyHow to do it...How it works...There's more…
Adding a force-directed graph of web hits
Getting readyHow to do it...How it works...There's more…Changing the time range on the search managerSee also
Adding a calendar heatmap of product purchases
Getting readyHow to do it...How it works...See also
Adding cell highlighting of average product price
Getting readyHow to do it...How it works...There's more…See also
Remotely querying Splunk's REST API for unique page views
Getting readyHow to do it...How it works...There's more…Authenticating with a session tokenSee also
Creating a Python application to return unique IP addresses
Getting readyHow to do it...How it works...There's more...Paginating the results of your searchSee also
Creating a custom search command to format product names
Getting readyHow to do it...How it works...See also
Collecting data from remote scanning devices
Getting readyHow to do it...How it works...See also
Index

Content preview from Splunk Operational Intelligence Cookbook - Second Edition

Finding the most accessed web pages

One of the data samples we loaded in Chapter 1, Play Time – Getting Data In, contained access logs from our web server. These have a Splunk sourcetype of access_combined and detail all pages accessed by the users of our web application. We are particularly interested in knowing which pages are being accessed the most, as this information provides great insight into how our e-commerce web application is being used. It could also help influence changes to our web application such that rarely visited pages are removed, or our application is redesigned to be more efficient.

In this recipe, we will write a Splunk search to find the most accessed web pages over a given period of time.

Getting ready

To step through this ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Splunk Operational Intelligence Cookbook

Publisher Resources

ISBN: 9781785284991

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Splunk Operational Intelligence Cookbook - Second Edition

by Josh Diakun, Jose E. Hernandez, Derek Mock, Paul R. Johnson

Finding the most accessed web pages

Getting ready

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.