Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Working with fields

All of the fields we have used so far were either indexed fields (such as host, sourcetype, and _time) or fields that were automatically extracted from key=value pairs. Unfortunately, most logs don't follow this format, especially for the first few values in each event. New fields can be created either inline, by using commands, or through configuration.

A regular expression primer

Most of the ways to create new fields in Splunk involve regular expressions. There are many books and sites dedicated to regular expressions, so we will only touch upon the subject here.

Given the log snippet ip=1.2.3.4, let's pull out the subnet (1.2.3) into a new field called subnet. The simplest pattern would be the literal string:

ip=(?P<subnet>1.2.3).4 ...