SiLK, the System for Internet-Level Knowledge, is a toolkit originally developed by Carnegie Mellon’s CERT to conduct large-scale NetFlow analysis. SiLK is now used extensively by the US Department of Defense, academic institutions, and technical companies as a basic analytical toolkit.
This chapter focuses primarily on using SiLK as an analytical tool. The CERT Network Situational Awareness (NetSA) Group has published extensive references on using SiLK, installing collectors, and setting up the suite.
SiLK is a suite of tools for querying and analyzing NetFlow data. The SiLK suite enables an analyst to rapidly and efficiently query very large volumes of network traffic in order to identify complex aggregate phenomena or extract individual events.
SiLK is effectively a database at the command line. Each tool
performs a specific query, manipulation, or aggregation of data, and
commands are chained together to produce results. By chaining
together multiple records along pipes, SiLK enables the analyst to
create complex commands that field data along multiple channels
simultaneously. For example, the sequence of SiLK queries
in Example 9-1 pulls HTTP (port 80) traffic from flow data, producing a time series
and a list of activity by busiest address. This example illustrates
the basics of SiLK operation: commands are passed through a series of
pipes, which can be
stdout, or FIFOs (named pipes).