Implementing Splunk 7 - Third Edition

Implementing Splunk 7 - Third Edition

by James D. Miller
Released March 2018
Publisher(s): Packt Publishing
ISBN: 9781788836289

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

A comprehensive guide to making machine data accessible across the organization using advanced dashboards

About This Book

  • Enrich machine-generated data and transform it into useful, meaningful insights
  • Perform search operations and configurations, build dashboards, and manage logs
  • Extend Splunk services with scripts and advanced configurations to process optimal results

Who This Book Is For

This book is intended for data analysts, business analysts, and IT administrators who want to make the best use of big data, operational intelligence, log management, and monitoring within their organization. Some knowledge of Splunk services will help you get the most out of the book

What You Will Learn

  • Focus on the new features of the latest version of Splunk Enterprise 7
  • Master the new offerings in Splunk: Splunk Cloud and the Machine Learning Toolkit
  • Create efficient and effective searches within the organization
  • Master the use of Splunk tables, charts, and graph enhancements
  • Use Splunk data models and pivots with faster data model acceleration
  • Master all aspects of Splunk XML dashboards with hands-on applications
  • Create and deploy advanced Splunk dashboards to share valuable business insights with peers

In Detail

Splunk is the leading platform that fosters an efficient methodology and delivers ways to search, monitor, and analyze growing amounts of big data. This book will allow you to implement new services and utilize them to quickly and efficiently process machine-generated big data.

We introduce you to all the new features, improvements, and offerings of Splunk 7. We cover the new modules of Splunk: Splunk Cloud and the Machine Learning Toolkit to ease data usage. Furthermore, you will learn to use search terms effectively with Boolean and grouping operators. You will learn not only how to modify your search to make your searches fast but also how to use wildcards efficiently. Later you will learn how to use stats to aggregate values, a chart to turn data, and a time chart to show values over time; you'll also work with fields and chart enhancements and learn how to create a data model with faster data model acceleration. Once this is done, you will learn about XML Dashboards, working with apps, building advanced dashboards, configuring and extending Splunk, advanced deployments, and more. Finally, we teach you how to use the Machine Learning Toolkit and best practices and tips to help you implement Splunk services effectively and efficiently.

By the end of this book, you will have learned about the Splunk software as a whole and implemented Splunk services in your tasks at projects

Style and approach

An easy-to-follow, step-by-step guide to help you get to grips with real-world applications of Splunk 7.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Implementing Splunk 7 Third Edition
  3. Packt Upsell
    1. Why subscribe?
    2. PacktPub.com
  4. Contributors
    1. About the author
    2. About the reviewer
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Get in touch
      1. Reviews
  6. The Splunk Interface
    1. Logging in to Splunk
    2. The home app
    3. The top bar
    4. The Search & Reporting app
      1. Data generator
      2. The Summary view
      3. Search
      4. Actions
      5. Timeline
      6. The field picker
        1. Fields
      7. Search results
        1. Options
        2. Events viewer
    5. Using the time picker
    6. Using the field picker
    7. The settings section
    8. Splunk Cloud
    9. Try before you buy
    10. A quick cloud tour
    11. The top bar in Splunk Cloud
    12. Splunk reference app – PAS
    13. Universal forwarder
    14. eventgen
    15. Next steps
    16. Summary
  7. Understanding Search
    1. Using search terms effectively
    2. Boolean and grouping operators
    3. Clicking to modify your search
      1. Event segmentation
      2. Field widgets
      3. Time
    4. Using fields to search
      1. Using the field picker
    5. Using wildcards efficiently
      1. Supplementing wildcards in fields
    6. All about time
      1. How Splunk parses time
      2. How Splunk stores time
      3. How Splunk displays time
      4. How time zones are determined and why it matters
      5. Different ways to search against time
        1. Presets
        2. Relative
        3. Real-time
          1. Windowed real-time versus all-time real-time searches
        4. Date range
        5. Date and time range
        6. Advanced
      6. Specifying time in-line in your search
        1. _indextime versus _time
    7. Making searches faster
    8. Sharing results with others
      1. The URL
      2. Save As Report
      3. Save As Dashboard Panel
      4. Save As Alert
      5. Save As Event Type
    9. Searching job settings
    10. Saving searches for reuse
    11. Creating alerts from searches
      1. Enable Actions
      2. Action Options
      3. Sharing
    12. Event annotations
      1. An illustration
    13. Summary
  8. Tables, Charts, and Fields
    1. About the pipe symbol
    2. Using top to show common field values
      1. Controlling the output of top
    3. Using stats to aggregate values
    4. Using chart to turn data
    5. Using timechart to show values over time
      1. The timechart options
    6. Working with fields
      1. A regular expression primer
      2. Commands that create fields
        1. eval
        2. rex
      3. Extracting loglevel
        1. Using the extract fields interface
        2. Using rex to prototype a field
        3. Using the admin interface to build a field
        4. Indexed fields versus extracted fields
          1. Indexed field case 1 - rare instances of a common term
          2. Indexed field case 2 - splitting words
          3. Indexed field case 3 - application from source
          4. Indexed field case 4 - slow requests
          5. Indexed field case 5 - unneeded work
    7. Chart enhancements in version 7.0
      1. charting.lineWidth
      2. charting.data.fieldHideList
      3. charting.legend.mode
      4. charting.fieldDashStyles
      5. charting.axis Y.abbreviation
    8. Summary
  9. Data Models and Pivots
    1. What is a data model?
    2. What does a data model search?
      1. Data model objects
        1. Object constraining
        2. Attributes
    3. Acceleration in version 7.0
    4. Creating a data model
      1. Filling in the new data model dialog
      2. Editing fields (attributes)
    5. Lookup attributes
      1. Children
    6. What is a pivot?
      1. The Pivot Editor
      2. Working with pivot elements
        1. Filtering pivots
      3. Split (row or column)
        1. Column values
      4. Pivot table formatting
    7. A quick example
    8. Sparklines
    9. Summary
  10. Simple XML Dashboards
    1. The purpose of dashboards
    2. Using wizards to build dashboards
      1. Adding another panel
        1. A cool trick
    3. Converting the panel to a report
      1. More options
    4. Back to the dashboard
      1. Add input
      2. Editing source
      3. Edit UI
    5. Editing XML directly
    6. UI examples app
    7. Building forms
      1. Creating a form from a dashboard
      2. Driving multiple panels from one form
      3. Post-processing search results
      4. Post-processing limitations
    8. Features replaced
    9. Autorun dashboard
    10. Scheduling the generation of dashboards
    11. Summary
  11. Advanced Search Examples
    1. Using subsearches to find loosely related events
      1. Subsearch
      2. Subsearch caveats
      3. Nested subsearches
    2. Using transaction
      1. Using transaction to determine session length
      2. Calculating the aggregate of transaction statistics
      3. Combining subsearches with transaction
    3. Determining concurrency
      1. Using transaction with concurrency
      2. Using concurrency to estimate server load
      3. Calculating concurrency with a by clause
    4. Calculating events per slice of time
      1. Using timechart
      2. Calculating average requests per minute
      3. Calculating average events per minute, per hour
    5. Rebuilding top
    6. Acceleration
      1. Big data – summary strategy
      2. Report acceleration
      3. Report acceleration availability
    7. Version 7.0 advancements in metrics
      1. Definition of a Splunk metric
      2. Using Splunk metrics
        1. Creating a metrics index
        2. Creating a UDP or TCP data input
    8. Summary
  12. Extending Search
    1. Using tags to simplify search
    2. Using event types to categorize results
    3. Using lookups to enrich data
      1. Defining a lookup table file
      2. Defining a lookup definition
      3. Defining an automatic lookup
      4. Troubleshooting lookups
    4. Using macros to reuse logic
      1. Creating a simple macro
      2. Creating a macro with arguments
    5. Creating workflow actions
      1. Running a new search using values from an event
      2. Linking to an external site
      3. Building a workflow action to show field context
        1. Building the context workflow action
        2. Building the context macro
    6. Using external commands
      1. Extracting values from XML
        1. xmlkv
        2. XPath
      2. Using Google to generate results
    7. Summary
  13. Working with Apps
    1. Defining an app
    2. Included apps
    3. Installing apps
      1. Installing apps from Splunkbase
        1. Using Geo Location Lookup Script
        2. Using Google Maps
      2. Installing apps from a file
    4. Building your first app
    5. Editing navigation
    6. Customizing the appearance of your app
      1. Customizing the launcher icon
      2. Using custom CSS
      3. Using custom HTML
        1. Custom HTML in a simple dashboard
        2. Using server-side include in a complex dashboard
    7. Object permissions
      1. How permissions affect navigation
      2. How permissions affect other objects
      3. Correcting permission problems
    8. App directory structure
      1. Adding your app to Splunkbase
        1. Preparing your app
        2. Confirming sharing settings
        3. Cleaning up our directories
      2. Packaging your app
      3. Uploading your app
    9. Self-service app management
    10. Summary
  14. Building Advanced Dashboards
    1. Reasons for working with advanced XML
    2. Reasons for not working with advanced XML
    3. Development process
    4. Advanced XML structure
    5. Converting simple XML to advanced XML
    6. Module logic flow
    7. Understanding layoutPanel
      1. Panel placement
    8. Reusing a query
    9. Using intentions
      1. stringreplace
      2. addterm
    10. Creating a custom drilldown
      1. Building a drilldown to a custom query
      2. Building a drilldown to another panel
      3. Building a drilldown to multiple panels using HiddenPostProcess
    11. Third-party add-ons
      1. Google Maps
      2. Sideview Utils
      3. The Sideview search module
        1. Linking views with Sideview
        2. Sideview URLLoader
        3. Sideview forms
    12. Summary
  15. Summary Indexes and CSV Files
    1. Understanding summary indexes
      1. Creating a summary index
    2. When to use a summary index
    3. When to not use a summary index
    4. Populating summary indexes with saved searches
    5. Using summary index events in a query
    6. Using sistats, sitop, and sitimechart
    7. How latency affects summary queries
    8. How and when to backfill summary data
      1. Using fill_summary_index.py to backfill
      2. Using collect to produce custom summary indexes
    9. Reducing summary index size
      1. Using eval and rex to define grouping fields
      2. Using a lookup with wildcards
      3. Using event types to group results
    10. Calculating top for a large time frame
      1. Summary index searches
    11. Using CSV files to store transient data
      1. Pre-populating a dropdown
      2. Creating a running calculation for a day
    12. Summary
  16. Configuring Splunk
    1. Locating Splunk configuration files
    2. The structure of a Splunk configuration file
    3. The configuration merging logic
      1. The merging order
        1. The merging order outside of search
        2. The merging order when searching
      2. The configuration merging logic
        1. Configuration merging – example 1
        2. Configuration merging – example 2
        3. Configuration merging – example 3
        4. Configuration merging – example 4, search
      3. Using btool
    4. An overview of Splunk.conf files
      1. props.conf
        1. Common attributes
          1. Search-time attributes
          2. Index-time attributes
          3. Parse-time attributes
          4. Input-time attributes
        2. Stanza types
        3. Priorities inside a type
        4. Attributes with class
      2. inputs.conf
        1. Common input attributes
        2. Files as inputs
          1. Using patterns to select rolled logs
          2. Using blacklist and whitelist
          3. Selecting files recursively
          4. Following symbolic links
          5. Setting the value of the host from the source
          6. Ignoring old data at installation
          7. When to use crcSalt
          8. Destructively indexing files
        3. Network inputs
        4. Native Windows inputs
        5. Scripts as inputs
      3. transforms.conf
        1. Creating indexed fields
          1. Creating a loglevel field
          2. Creating a session field from the source
          3. Creating a tag field
          4. Creating host categorization fields
        2. Modifying metadata fields
          1. Overriding the host
          2. Overriding the source
          3. Overriding sourcetype
          4. Routing events to a different index
        3. Lookup definitions
          1. Wildcard lookups
          2. CIDR wildcard lookups
          3. Using time in lookups
        4. Using REPORT
          1. Creating multivalue fields
          2. Creating dynamic fields
        5. Chaining transforms
        6. Dropping events
      4. fields.conf
      5. outputs.conf
      6. indexes.conf
      7. authorize.conf
      8. savedsearches.conf
      9. times.conf
      10. commands.conf
      11. web.conf
    5. User interface resources
      1. Views and navigation
      2. Appserver resources
      3. Metadata
    6. Summary
  17. Advanced Deployments
    1. Planning your installation
    2. Splunk instance types
      1. Splunk forwarders
      2. Splunk indexer
      3. Splunk search
    3. Common data sources
      1. Monitoring logs on servers
      2. Monitoring logs on a shared drive
      3. Consuming logs in batch
      4. Receiving syslog events
        1. Receiving events directly on the Splunk indexer
        2. Using a native syslog receiver
        3. Receiving syslog with a Splunk forwarder
      5. Consuming logs from a database
      6. Using scripts to gather data
    4. Sizing indexers
    5. Planning redundancy
      1. The replication factor
        1. Configuring your replication factors
          1. Syntax
      2. Indexer load balancing
      3. Understanding typical outages
    6. Working with multiple indexes
      1. Directory structure of an index
      2. When to create more indexes
        1. Testing data
        2. Differing longevity
        3. Differing permissions
        4. Using more indexes to increase performance
      3. The life cycle of a bucket
      4. Sizing an index
      5. Using volumes to manage multiple indexes
    7. Deploying the Splunk binary
      1. Deploying from a tar file
      2. Deploying using msiexec
      3. Adding a base configuration
      4. Configuring Splunk to launch at boot
    8. Using apps to organize configuration
      1. Separate configurations by purpose
    9. Configuration distribution
      1. Using your own deployment system
      2. Using the Splunk deployment server
        1. Step 1 – deciding where your deployment server will run
        2. Step 2 - defining your deploymentclient.conf configuration
        3. Step 3 - defining our machine types and locations
        4. Step 4 - normalizing our configurations into apps appropriately
        5. Step 5 - mapping these apps to deployment clients in serverclass.conf
        6. Step 6 - restarting the deployment server
        7. Step 7 - installing deploymentclient.conf
    10. Using LDAP for authentication
    11. Using single sign-on
    12. Load balancers and Splunk
      1. web
      2. splunktcp
      3. deployment server
    13. Multiple search heads
    14. Summary
  18. Extending Splunk
    1. Writing a scripted input to gather data
      1. Capturing script output with no date
      2. Capturing script output as a single event
      3. Making a long-running scripted input
    2. Using Splunk from the command line
    3. Querying Splunk via REST
    4. Writing commands
      1. When not to write a command
      2. When to write a command
      3. Configuring commands
      4. Adding fields
      5. Manipulating data
      6. Transforming data
      7. Generating data
    5. Writing a scripted lookup to enrich data
    6. Writing an event renderer
      1. Using specific fields
      2. A table of fields based on field value
      3. Pretty printing XML
    7. Writing a scripted alert action to process results
    8. Hunk
    9. Summary
  19. Machine Learning Toolkit
    1. What is machine learning?
      1. Content recommendation engines
      2. Natural language processing
      3. Operational intelligence
    2. Defining the toolkit
      1. Time well spent
      2. Obtaining the Kit
        1. Prerequisites and requirements
        2. Installation
    3. The toolkit workbench
    4. Assistants
    5. Extended SPL (search processing language)
      1. ML-SPL performance app
    6. Building a model
      1. Time series forecasting
      2. Using Splunk
      3. Launching the toolkit
    7. Validation
      1. Deployment
      2. Saving a report
      3. Exporting data
    8. Summary

Product information

  • Title: Implementing Splunk 7 - Third Edition
  • Author(s): James D. Miller
  • Release date: March 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781788836289