Mastering Linux Shell Scripting - Second Edition

Summarizing Logs with AWK

In the previous chapter, we talked about regular expressions and we saw how to use them to empower sed and AWK. In this chapter, we will discuss some practical examples of using AWK.

One of the tasks that AWK is really good at is filtering data from log files. These log files may be many lines in length, perhaps 250,000 or more. I have worked with data with over a million lines. AWK can process these lines quickly and effectively. As an example, we will work with a web server access log with 30,000 lines to show how effective and well-written AWK code can be. As we work our way through the chapter, we will also see different log files and review some of the techniques that we can employ with the awk command and the ...