Skip to Main Content
Regular Expressions Cookbook, 2nd Edition
book

Regular Expressions Cookbook, 2nd Edition

by Jan Goyvaerts, Steven Levithan
August 2012
Intermediate to advanced content levelIntermediate to advanced
609 pages
19h 16m
English
O'Reilly Media, Inc.
Content preview from Regular Expressions Cookbook, 2nd Edition

Common Log Format

Problem

You need a regular expression that matches each line in the log files produced by a web server that uses the Common Log Format.[11] For example:

127.0.0.1 - jg [27/Apr/2012:11:27:36 +0700] "GET /regexcookbook.html HTTP/1.1" 200 2326

The regular expression should have a capturing group for each field, to allow the application using the regular expression to easily process the fields of each entry in the log.

Solution

^(?<client>\S+)\S+(?<userid>\S+)\[(?<datetime>[^\]]+)\]↵
"(?<method>[A-Z]+)(?<request>[^"]+)?HTTP/[0-9.]+"↵
(?<status>[0-9]{3})(?<size>[0-9]+|-)
Regex options: ^ and $ match at line breaks
Regex flavors: .NET, Java 7, XRegExp, PCRE 7, Perl 5.10, Ruby 1.9
^(?P<client>\S+)\S+(?P<userid>\S+)\[(?P<datetime>[^\]]+)\]↵
"(?P<method>[A-Z]+)(?P<request>[^"]+)?HTTP/[0-9.]+"↵
(?P<status>[0-9]{3})(?P<size>[0-9]+|-)
Regex options: ^ and $ match at line breaks
Regex flavors: PCRE 4, Perl 5.10, Python
^(\S+)\S+(\S+)\[([^\]]+)\]"([A-Z]+)([^"]+)?HTTP/[0-9.]+"↵
([0-9]{3})([0-9]+|-)"([^"]*)""([^"]*)"
Regex options: ^ and $ match at line breaks
Regex flavors: .NET, Java, JavaScript, PCRE, Perl, Python, Ruby

Discussion

Creating a regular expressions to match any entry in a log file generally is very straightforward. It certainly is when the log format puts the same information in each entry, just with different values. This is true for web servers that save access logs using the Common Log Format, such as Apache. Each line in the log file is one log ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Mastering Regular Expressions, 3rd Edition

Mastering Regular Expressions, 3rd Edition

Jeffrey E.F. Friedl
Regular Expressions Cookbook

Regular Expressions Cookbook

Jan Goyvaerts, Steven Levithan

Publisher Resources

ISBN: 9781449327453Supplemental ContentErrata Page