Linux Security Cookbook

9.35. Combining Log Files

Problem

You want to merge a collection of log files into a single, chronological log file.

Solution

#!/bin/sh
perl -ne \
   'print $last, /last message repeated \d+ times$/ ? "\0" : "\n" if $last;
    chomp($last = $_);    
    if (eof) {
        print;
        undef $last;
    }' "$@" | sort -s -k 1,1M -k 2,2n -k 3,3 | tr '\0' '\n'

Discussion

The system logger automatically prepends a timestamp to each message, like this:

Feb 21 12:34:56 buster kernel: device eth0 entered promiscuous mode

To merge log files, sort each one by its timestamp entries, using the first three fields (month, date, and time) as keys.

A complication arises because the system logger inserts “repetition messages” to conserve log file space:

Feb 21 12:48:16 buster last message repeated 7923 times

The timestamp for the repetition message is often later than the last message. It would be terribly misleading if possibly unrelated messages from other log files were merged between the last message and its associated repetition message.

To avoid this, our Perl script glues together the last message with a subsequent repetition message (if present), inserting a null character between them: this is reliable because the system logger never writes null characters to log files. The script writes out the final line before the end of each file and then forgets the last line, to avoid any possibility of confusion if the next file happens to start with an unrelated repetition message.

The sort command sees these null-glued combinations as ...