Skip to Main Content
Perl Cookbook
book

Perl Cookbook

by Tom Christiansen, Nathan Torkington
August 1998
Intermediate to advanced content levelIntermediate to advanced
800 pages
39h 20m
English
O'Reilly Media, Inc.
Content preview from Perl Cookbook

Parsing a Web Server Log File

Problem

You want to extract from a web server log file only the information you’re interested in.

Solution

Pull apart the log file as follows:

while (<LOGFILE>) {
  my ($client, $identuser, $authuser, $date, $time, $tz, $method,
      $url, $protocol, $status, $bytes) =
  /^(\S+) (\S+) (\S+) \[([^:]+):(\d+:\d+:\d+) ([^\]]+) "(\S+) (.*?) (\S+)"
      (\S+) (\S+)$/;
  # ...
}

Discussion

This regular expression pulls apart entries in Common Log Format, an informal standard that most web servers adhere to. The fields are:

client

IP address or domain name of browser’s machine

identuser

If IDENT (RFC 1413) was used, what it returned

authuser

If username/password authentication was used, whom they logged in as

date

Date of request (e.g., 01/Mar/1997)

time

Time of request (e.g., 12:55:36)

tz

Time zone (e.g., -0700)

method

Method of request (e.g., GET, POST, or PUT)

url

URL in request (e.g., /~user/index.html)

protocol

HTTP/1.0 or HTTP/1.1

status

Returned status (200 is okay, 500 is server error)

bytes

Number of bytes returned (could be "-" for errors, redirects, and other non-document transfers)

Other formats include the referrer and agent information. The pattern needs only minor changes for it to work with other log file formats. Watch out that spaces in the URL field are not escaped. This means that we can’t use \S* to extract the URL. .* would cause the regex to match the entire string and then backtrack until it could satisfy the rest of the pattern. We use .*? and anchor the pattern to the end ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Perl in a Nutshell

Perl in a Nutshell

Nathan Patwardhan, Ellen Siever, Stephen Spainhour
Perl Best Practices

Perl Best Practices

Damian Conway
Mastering Perl

Mastering Perl

brian d foy
Perl Cookbook, 2nd Edition

Perl Cookbook, 2nd Edition

Tom Christiansen, Nathan Torkington

Publisher Resources

ISBN: 1565922433Supplemental ContentCatalog PageErrata