The Practice of Network Security Monitoring

Table of Contents

1. Dedication
2. Foreword
3. Preface
   1. Audience
   2. Prerequisites
   3. A Note on Software and Protocols
   4. Scope
   5. Acknowledgments
   6. Disclaimer
4. I. Getting Started
   1. 1. Network Security Monitoring Rationale
      1. An Introduction to NSM
         1. Does NSM Prevent Intrusions?
         2. What Is the Difference Between NSM and Continuous Monitoring?
         3. How Does NSM Compare with Other Approaches?
         4. Why Does NSM Work?
         5. How NSM Is Set Up
            1. Installing a Tap
         6. When NSM Won’t Work
         7. Is NSM Legal?
         8. How Can You Protect User Privacy During NSM Operations?
      2. A Sample NSM Test
      3. The Range of NSM Data
         1. Full Content Data
            1. Reviewing a Data Summary
            2. Inspecting Packets
            3. Using a Graphical Tool to View the Traffic
         2. Extracted Content Data
         3. Session Data
         4. Transaction Data
         5. Statistical Data
         6. Metadata
         7. Alert Data
      4. What’s the Point of All This Data?
      5. NSM Drawbacks
      6. Where Can I Buy NSM?
      7. Where Can I Go for Support or More Information?
      8. Conclusion
   2. 2. Collecting Network Traffic: Access, Storage, and Management
      1. A Sample Network for a Pilot NSM System
         1. Traffic Flow in a Simple Network
         2. Possible Locations for NSM
      2. IP Addresses and Network Address Translation
         1. Net Blocks
         2. IP Address Assignments
         3. Address Translation
            1. Network Address Translation
            2. Address Translation in Wireless and Internal Networks
      3. Choosing the Best Place to Obtain Network Visibility
         1. Location for DMZ Network Traffic
         2. Locations for Viewing the Wireless and Internal Network Traffic
      4. Getting Physical Access to the Traffic
         1. Using Switches for Traffic Monitoring
         2. Using a Network Tap
         3. Capturing Traffic Directly on a Client or Server
      5. Choosing an NSM Platform
      6. Ten NSM Platform Management Recommendations
      7. Conclusion
5. II. Security Onion Deployment
   1. 3. Stand-alone NSM Deployment and Installation
      1. Stand-alone or Server Plus Sensors?
      2. Choosing How to Get SO Code onto Hardware
      3. Installing a Stand-alone System
         1. Installing SO to a Hard Drive
         2. Configuring SO Software
         3. Choosing the Management Interface
         4. Installing the NSM Software Components
         5. Checking Your Installation
      4. Conclusion
   2. 4. Distributed Deployment
      1. Installing an SO Server Using the SO .iso Image
         1. SO Server Considerations
         2. Building Your SO Server
         3. Configuring Your SO Server
      2. Installing an SO Sensor Using the SO .iso Image
         1. Configuring the SO Sensor
         2. Completing Setup
         3. Verifying that the Sensors Are Working
         4. Verifying that the Autossh Tunnel Is Working
      3. Building an SO Server Using PPAs
         1. Installing Ubuntu Server as the SO Server Operating System
         2. Choosing a Static IP Address
         3. Updating the Software
         4. Beginning MySQL and PPA Setup on the SO Server
         5. Configuring Your SO Server via PPA
      4. Building an SO Sensor Using PPAs
         1. Installing Ubuntu Server as the SO Sensor Operating System
         2. Configuring the System as a Sensor
         3. Running the Setup Wizard
      5. Conclusion
   3. 5. SO Platform Housekeeping
      1. Keeping SO Up-to-Date
         1. Updating via the GUI
         2. Updating via the Command Line
      2. Limiting Access to SO
         1. Connecting via a SOCKS Proxy
         2. Changing the Firewall Policy
      3. Managing SO Data Storage
         1. Managing Sensor Storage
         2. Checking Database Drive Usage
         3. Managing the Sguil Database
         4. Tracking Disk Usage
      4. Conclusion
6. III. Tools
   1. 6. Command Line Packet Analysis Tools
      1. SO Tool Categories
         1. SO Data Presentation Tools
            1. Packet Analysis Tools
            2. NSM Consoles
         2. SO Data Collection Tools
         3. SO Data Delivery Tools
      2. Running Tcpdump
         1. Displaying, Writing, and Reading Traffic with Tcpdump
         2. Using Filters with Tcpdump
            1. Applying Filters
            2. Some Common Filters
         3. Extracting Details from Tcpdump Output
         4. Examining Full Content Data with Tcpdump
      3. Using Dumpcap and Tshark
         1. Running Tshark
         2. Running Dumpcap
         3. Running Tshark on Dumpcap’s Traffic
         4. Using Display Filters with Tshark
         5. Tshark Display Filters in Action
      4. Running Argus and the Ra Client
         1. Stopping and Starting Argus
         2. The Argus File Format
         3. Examining Argus Data
      5. Conclusion
   2. 7. Graphical Packet Analysis Tools
      1. Using Wireshark
         1. Running Wireshark
         2. Viewing a Packet Capture in Wireshark
         3. Modifying the Default Wireshark Layout
            1. Modifying the Layout Using the GUI
            2. Modifying the Preferences File
         4. Some Useful Wireshark Features
            1. Viewing Lower-Level Protocol Features in Detail
            2. Omitting Traffic to See Remnants
            3. Following Streams
            4. Setting the Protocol Decode Method with Decode As
            5. Following Other Streams
      2. Using Xplico
         1. Running Xplico
         2. Creating Xplico Cases and Sessions
         3. Processing Network Traffic
         4. Understanding the Decoded Traffic
         5. Getting Metadata and Summarizing Traffic
      3. Examining Content with NetworkMiner
         1. Running NetworkMiner
         2. Collecting and Organizing Traffic Details
         3. Rendering Content
      4. Conclusion
   3. 8. NSM Consoles
      1. An NSM-centric Look at Network Traffic
      2. Using Sguil
         1. Running Sguil
         2. Sguil’s Six Key Functions
            1. Simple Aggregation
            2. Metadata and Related Data
            3. Querying Alert Data in Sguil
            4. Querying Session Data in Sguil
            5. Pivoting to Full Content Data
            6. Categorizing Alert Data
      3. Using Squert
      4. Using Snorby
      5. Using ELSA
      6. Conclusion
7. IV. NSM in Action
   1. 9. NSM Operations
      1. The Enterprise Security Cycle
         1. The Planning Phase
         2. The Resistance Phase
         3. The Detection and Response Phases
      2. Collection, Analysis, Escalation, and Resolution
         1. Collection
            1. Technical Sources
            2. Nontechnical Sources
         2. Analysis
            1. Intrusions and Incidents
            2. Event Classification
         3. Escalation
            1. Documentation of Incidents
            2. Notification of Incidents
            3. Incident Communication Considerations
         4. Resolution
            1. Containment Techniques
            2. Speed of Containment
      3. Remediation
         1. Using NSM to Improve Security
         2. Building a CIRT
      4. Conclusion
   2. 10. Server-side Compromise
      1. Server-side Compromise Defined
      2. Server-side Compromise in Action
         1. Starting with Sguil
         2. Querying Sguil for Session Data
         3. Returning to Alert Data
         4. Reviewing Full Content Data with Tshark
         5. Understanding the Backdoor
         6. What Did the Intruder Do?
            1. Initial Access
            2. Enumerating the Victim
            3. Accessing Credentials
         7. What Else Did the Intruder Do?
      3. Exploring the Session Data
         1. Searching Bro DNS Logs
         2. Searching Bro SSH Logs
         3. Searching Bro FTP Logs
         4. Decoding the Theft of Sensitive Data
         5. Extracting the Stolen Archive
      4. Stepping Back
         1. Summarizing Stage 1
         2. Summarizing Stage 2
         3. Next Steps
      5. Conclusion
   3. 11. Client-side Compromise
      1. Client-side Compromise Defined
      2. Client-side Compromise in Action
         1. Getting the Incident Report from a User
         2. Starting Analysis with ELSA
            1. Querying for the IP Address
            2. Checking the Bro HTTP Log
            3. Checking Snort Alerts
            4. Searching for Other Activity
         3. Looking for Missing Traffic
      3. Analyzing the Bro dns.log File
      4. Checking Destination Ports
      5. Examining the Command-and-Control Channel
         1. Initial Access
         2. Improving the Shell
         3. Summarizing Stage 1
         4. Pivoting to a Second Victim
         5. Installing a Covert Tunnel
         6. Enumerating the Victim
         7. Summarizing Stage 2
      6. Conclusion
   4. 12. Extending SO
      1. Using Bro to Track Executables
         1. Hashing Downloaded Executables with Bro
         2. Submitting a Hash to VirusTotal
      2. Using Bro to Extract Binaries from Traffic
         1. Configuring Bro to Extract Binaries from Traffic
         2. Collecting Traffic to Test Bro
         3. Testing Bro to Extract Binaries from HTTP Traffic
         4. Examining the Binary Extracted from HTTP
         5. Testing Bro to Extract Binaries from FTP Traffic
         6. Examining the Binary Extracted from FTP
         7. Submitting a Hash and Binary to VirusTotal
         8. Restarting Bro
      3. Using APT1 Intelligence
         1. Using the APT1 Module
         2. Installing the APT1 Module
         3. Generating Traffic to Test the APT1 Module
         4. Testing the APT1 Module
      4. Reporting Downloads of Malicious Binaries
         1. Using the Team Cymru Malware Hash Registry
         2. The MHR and SO: Active by Default
         3. The MHR and SO vs. a Malicious Download
         4. Identifying the Binary
      5. Conclusion
   5. 13. Proxies and Checksums
      1. Proxies
         1. Proxies and Visibility
            1. Traffic from the Client to the Proxy
            2. Traffic from the Proxy to the Web Server
         2. Dealing with Proxies in Production Networks
      2. Checksums
         1. A Good Checksum
         2. A Bad Checksum
         3. Identifying Bad and Good Checksums with Tshark
         4. How Bad Checksums Happen
         5. Bro and Bad Checksums
         6. Setting Bro to Ignore Bad Checksums
      3. Conclusion
8. Conclusion
   1. Cloud Computing
      1. Cloud Computing Challenges
      2. Cloud Computing Benefits
   2. Workflow, Metrics, and Collaboration
      1. Workflow and Metrics
      2. Collaboration
   3. Conclusion
9. A. SO Scripts and Configuration
   1. SO Control Scripts
      1. /usr/sbin/nsm
      2. /usr/sbin/nsm_all_del
      3. /usr/sbin/nsm_all_del_quick
      4. /usr/sbin/nsm_sensor
      5. /usr/sbin/nsm_sensor_add
      6. /usr/sbin/nsm_sensor_backup-config
      7. /usr/sbin/nsm_sensor_backup-data
      8. /usr/sbin/nsm_sensor_clean
      9. /usr/sbin/nsm_sensor_clear
      10. /usr/sbin/nsm_sensor_del
      11. /usr/sbin/nsm_sensor_edit
      12. /usr/sbin/nsm_sensor_ps-daily-restart
      13. /usr/sbin/nsm_sensor_ps-restart
      14. /usr/sbin/nsm_sensor_ps-start
      15. /usr/sbin/nsm_sensor_ps-status
      16. /usr/sbin/nsm_sensor_ps-stop
      17. /usr/sbin/nsm_server
      18. /usr/sbin/nsm_server_add
      19. /usr/sbin/nsm_server_backup-config
      20. /usr/sbin/nsm_server_backup-data
      21. /usr/sbin/nsm_server_clear
      22. /usr/sbin/nsm_server_del
      23. /usr/sbin/nsm_server_edit
      24. /usr/sbin/nsm_server_ps-restart
      25. /usr/sbin/nsm_server_ps-start
      26. /usr/sbin/nsm_server_ps-status
      27. /usr/sbin/nsm_server_ps-stop
      28. /usr/sbin/nsm_server_sensor-add
      29. /usr/sbin/nsm_server_sensor-del
      30. /usr/sbin/nsm_server_user-add
   2. SO Configuration Files
      1. /etc/nsm/
      2. /etc/nsm/administration.conf
      3. /etc/nsm/ossec/
      4. /etc/nsm/pulledpork/
      5. /etc/nsm/rules/
      6. /etc/nsm/securityonion/
      7. /etc/nsm/securityonion.conf
      8. /etc/nsm/sensortab
      9. /etc/nsm/servertab
      10. /etc/nsm/templates/
      11. /etc/nsm/$HOSTNAME-$INTERFACE/
          1. barnyard2.conf
          2. bpf.conf files
          3. http_agent.conf
          4. pads_agent.conf
          5. pcap_agent.conf
          6. prads.conf
          7. sancp_agent.conf
          8. sensor.conf
          9. snort_agent.conf
          10. snort.conf
          11. suricata.yaml
      12. /etc/cron.d/
      13. Bro
      14. CapMe
      15. ELSA
      16. Squert
      17. Snorby
      18. Syslog-ng
      19. /etc/network/interfaces
   3. Updating SO
      1. Updating the SO Distribution
      2. Updating MySQL
10. Index
11. About the Author
12. Colophon
13. B. Updates
14. Copyright