Splunk Best Practices

Anatomy of an alert

There are some very fundamental parts of an alert that are generic to any alerting system. They are translatable to Nagios, SCOM, Icinga, or take your pick. In Splunk, however, there are some unique components of an alert that give us the ability to enhance what the alert itself does, and mostly it has to do with SPL(Splunk Processing Language). Once we have gotten the results we want, there are some fun things we can do with an alert.

Search query results

This is the result set of any search that you determine viable for an alert. It is often easiest to use a stats command to set an alert, as it gives an integer that can easily be filtered by a where statement. The amount of history searched is also very important in the setup ...