Network Security Hacks, 2nd Edition

Chapter 9. Monitoring and Trending

Hacks 87–91

While the importance of reliable system logs can’t be overestimated, logs tell only part of the story of what is happening on your network. When something out of the ordinary happens, the event is duly logged to the appropriate file, where it waits for a human to notice and take the appropriate action. But logs are valuable only if someone actually reads them. When log files simply add to the deluge of information that most network administrators must wade through each day, they might be put aside and go unread for days or weeks. This situation is made worse when the log files are clogged with irrelevant information. For example, a cry for help from an overburdened mail server can easily be lost if it is surrounded by innocuous entries about failed spam attempts. All too often, logs are used as a resource to figure out what happened when systems fail, rather than as a guide to what is happening now.

Another important aspect of log entries is that they only provide a “spot check” of your system at a particular moment. Without a history of what normal performance looks like, it can be difficult to tell the difference between ordinary network traffic, a denial-of-service (DoS) attack, and a visitation from Slashdot readers. While you can easily build a report on how many times the /var partition filled up, how can you track what normal usage looks like over time? Is the mail spool clogged due to one inconsiderate user, or is it part of ...