It is never impossible for a hacker to break into a computer system, only improbable.
Computer hackers routinely break into corporate, military, online banking, and other networked environments. Even in 2007, as I am writing this second edition of Network Security Assessment, I still perform incident response work in these sectors. As systems generally become more secure, the methods used by these attackers are becoming more advanced, involving intricate repositioning, social engineering, physical compromise (stealing disks from servers or installing rogue wireless access points), and use of specific zero-day exploits to attack peripheral software components such as antivirus or backup solutions that are widely deployed internally within corporate networks.
By the same token, you would expect professional security consultants to be testing for these types of issues. In the vast majority of cases they are not. I know this because at Matta we run a program called Sentinel, which involves testing security assessment vendors for companies in the financial services sector. The Sentinel platform contains a number of vulnerable systems, and vendors are scored based on the vulnerabilities they identify and report.
Since 2004, Matta has processed nearly 30 global penetration testing vendors using Sentinel. In a recent test involving 10 testing providers, we found the following:
Two vendors failed to scan all 65536 TCP ports
Five vendors failed to report the publicly accessible MySQL service root password of “password”
Seven vendors failed to report the easily exploitable, high-risk SSL PCT overflow (MS04-011)
A number of vendors have tested the Sentinel platform on more than one occasion. It is clear that there is a lack of adherence to a strict testing methodology, and test results (in particular, the final report presented to the customer) vary wildly, depending on the consultant involved.
So here I am, in 2007, updating this book with a clear vision: to document a clear and concise Internet-based network security assessment methodology and approach. After running the Sentinel program through a number of iterations, performing a number of challenging penetration tests myself, and working to build a competent team at Matta, I feel it is the right time to update this book.
This book tackles one single area of information security in detail: that of undertaking IP-based network security assessment in a structured and logical way. The methodology presented in this book describes how a determined attacker will scour Internet-based networks in search of vulnerable components (from the network to the application level) and how you can perform exercises to assess your networks effectively. This book doesn’t contain any information that isn’t relevant to IP-based security testing; topics that are out of scope include war dialing and 802.11 wireless assessment.
Assessment is the first step any organization should take to start managing information risks correctly. My background is that of a teenage hacker turned professional security analyst, with a 100 percent success rate over the last nine years in compromising the networks of multinational corporations. I have a lot of fun working in the security industry and feel that now is the time to start helping others by clearly defining an effective best-practice network assessment methodology.
By assessing your networks in the same way that a determined attacker does, you can take a more proactive approach to risk management. Throughout this book, there are bulleted checklists of countermeasures to help you devise a clear technical strategy and fortify your environments at the network and application levels.
Recognized Assessment Standards
This book has been written in line with government penetration testing standards used in the United States (NSA IAM) and the United Kingdom (CESG CHECK). Other testing standards associations include MasterCard SDP, CREST, CEH, and OSSTMM. These popular accreditation programs are discussed here.
The United States National Security Agency (NSA) has provided an INFOSEC Assessment Methodology (IAM) framework to help consultants and security professionals outside the NSA provide assessment services to clients in line with a recognized standard. The NSA IAM home page is http://www.iatrp.com.
The IAM framework defines three levels of assessment related to the testing of IP-based computer networks:
Level 1 involves discovering a cooperative high-level overview of the organization being assessed, including access to policies, procedures, and information flow. No hands-on network or system testing is undertaken at this level.
Level 2 is a hands-on cooperative process that involves testing with network scanning, penetration tools, and the use of specific technical expertise.
- Red Team
Level 3 is noncooperative and external to the target network, involving penetration testing to simulate the appropriate adversary. IAM assessment is nonintrusive, so within this framework, a Level 3 assessment involves full qualification of vulnerabilities.
This book covers only the technical network scanning and assessment techniques used within Levels 2 (Evaluation) and 3 (Red Team) of the IAM framework, since Level 1 assessment involves high-level cooperative gathering of information, such as security policies.
The Government Communications Headquarters (GCHQ) in the United Kingdom has an information assurance arm known as the Communications and Electronics Security Group (CESG). In the same way that the NSA IAM framework allows security consultants outside the NSA to provide assessment services, CESG operates a program known as CHECK to evaluate and accredit security testing teams within the U.K. to undertake government assessment work. The CESG CHECK home page is accessible at http://www.cesg.gov.uk/site/check/index.cfm.
Unlike the NSA IAM, which covers many aspects of information security (including review of security policy, antivirus, backups, and disaster recovery), CHECK squarely tackles the area of network security assessment. A second program is the CESG Listed Adviser Scheme (CLAS), which covers information security in a broader sense and tackles areas such as ISO/IEC 27002, security policy creation, and auditing.
To correctly accredit CHECK consultants, CESG runs an assault course to test the attack and penetration techniques and methods demonstrated by attendees. The unclassified CESG CHECK assault course lists the areas of technical competence relating to network security assessment as:
Use of DNS information retrieval tools for both single and multiple records, including an understanding of DNS record structure relating to target hosts
Use of ICMP, TCP, and UDP network mapping and probing tools
Demonstration of TCP service banner grabbing
Information retrieval using SNMP, including an understanding of MIB structure relating to target system configuration and network routes
Understanding of common weaknesses in routers and switches relating to Telnet, HTTP, SNMP, and TFTP access and configuration
The following are Unix-specific competencies:
User enumeration via finger, rusers, rwho, and SMTP techniques
Use of tools to enumerate Remote Procedure Call (RPC) services and demonstrate an understanding of the security implications associated with those services
Demonstration of testing for Network File System (NFS) weaknesses
Testing for weaknesses within r-services (rsh, rexec, and rlogin)
Detection of insecure X Windows servers
Testing for weaknesses within web, FTP, and Samba services
Here are Windows NT-specific competencies:
Assessment of NetBIOS and CIFS services to enumerate users, groups, shares, domains, domain controllers, password policies, and associated weaknesses
Username and password grinding via NetBIOS and CIFS services
Detecting and demonstrating presence of known security weaknesses within Internet Information Server (IIS) web and FTP service components, and Microsoft SQL Server
This book clearly documents assessments in all these listed areas, along with background information to help you gain a sound understanding of the vulnerabilities presented. Although the CESG CHECK program assesses the methodologies of consultants who wish to perform U.K. government security testing work, internal security teams of organizations and companies outside the United Kingdom should be aware of its framework and common body of knowledge.
PCI Data Security Standards
Two security assessment accreditations that have gained popularity in recent years are the MasterCard Site Data Protection (SDP) program, which, along with the VISA Account Information Security (AIS) scheme, form Payment Card Industry (PCI) data security standards. Merchants, processors, and data storage entities that process payment card data must be assessed by a PCI-compliant vendor. The PCI accreditation program assault course is similar to that operated under CESG CHECK and Matta Sentinel, in that consultants must test a network of vulnerable servers and devices, and must accurately find and report the seeded vulnerabilities.
Further details of the PCI data security standards, the MasterCard SDP program, and VISA AIS are available from the following sites:
Other Assessment Standards and Associations
ISECOM’s Open Source Security Testing Methodology Manual (OSSTMM) (http://www.osstmm.org)
Council of Registered Ethical Security Testers (CREST) (http://www.crestapproved.com)
TIGER Scheme (http://www.tigerscheme.org)
EC-Council’s Certified Ethical Hacker (CEH) (http://www.eccouncil.org/ceh.htm)
Open Source Web Application Security Project (OWASP) (http://www.owasp.org)
The art of manipulating a process in such a way that it performs an action that is useful to you.
I think this is a true representation of a hacker in any sense of the word, whether it be a computer programmer who used to hack code on mainframes back in the day so that it would perform actions useful to him, or a modern computer attacker with a very different goal and set of ethics. Please bear in mind that when I use the term hacker in this book, I am talking about a network-based assailant trying to compromise the security of a system. I don’t mean to step on the toes of hackers in the traditional sense who have sound ethics and morals.
This book consists of 16 chapters and 3 appendixes. At the end of each chapter is a checklist that summarizes the threats and techniques described in that chapter along with effective countermeasures. The appendixes provide useful reference material, including listings of TCP and UDP ports, along with ICMP message types and their functions. Details of popular vulnerabilities in Microsoft Windows and Unix-based operating platforms are also listed. Here is a brief description of each chapter and appendix:
Chapter 1, discusses the rationale behind network security assessment and introduces security as a process, not a product.
Chapter 2, covers the various operating systems and tools that make up a professional security consultant’s attack platform.
Chapter 3, logically walks through the Internet-based options that a potential attacker has to map your network, from open web searches to DNS sweeping and querying of authoritative name servers.
Chapter 4, discusses all known IP network scanning techniques and their relevant applications, also listing tools and systems that support such scanning types. IDS evasion and low-level packet analysis techniques are also covered.
Chapter 5, defines the techniques and tools that execute information leak attacks against services such as LDAP, finger, and DNS. Some process manipulation attacks are discussed here when appropriate.
Chapter 6, covers the assessment of underlying web services, including Microsoft IIS, Apache, Tomcat, and subsystems such as OpenSSL, Microsoft FrontPage, and Outlook Web Access (OWA).
Chapter 7, covers assessment of various web application technologies, including ASP, JSP, PHP, middleware, and backend databases such as MySQL, Oracle, and Microsoft SQL Server. Also covered here is the use of tools such as Paros and WebScarab.
Chapter 8, details the tools and techniques used to correctly assess all common maintenance services (including FTP, SSH, VNC, X Windows, and Microsoft Terminal Services). Increasingly, these services are targets of information leak and brute-force attacks, resulting in a compromise even though the underlying software isn’t strictly vulnerable.
Chapter 9, covers IP-based assessment of database servers including Oracle, Microsoft SQL Server, and MySQL.
Chapter 10, tackles security assessment for Windows components (including MSRPC, NetBIOS, and CIFS) in a port-by-port fashion. Information leak, brute-force, and process manipulation attacks against each component are detailed, from the DCE locator service listening on port 135 through to the CIFS direct listener on port 445.
Chapter 11, details assessment of SMTP, POP-3, and IMAP services that transport email. Often, these services can fall foul to information-leak and brute-force attacks, and, in some instances, process manipulation.
Chapter 12, covers assessment of IP services that provide secure inbound network access, including IPsec, Microsoft PPTP, and SSL VPNs.
Chapter 13, comprehensively covers assessment of Unix RPC services found running on Linux, Solaris, IRIX, and other platforms. RPC services are commonly abused to gain access to hosts, so it is imperative that any accessible services are correctly assessed.
Chapter 14, defines the various types of application-level vulnerabilities that hacker tools and scripts exploit. By grouping vulnerabilities in this way, a timeless risk management model can be realized because all future application-level risks will fall into predefined groups.
Chapter 15, details how to set up and configure the Nessus vulnerability scanner to perform effective and fast automated testing of networks.
Chapter 16, covers the selection and use of exploitation frameworks, including the Metasploit Framework (MSF), Immunity CANVAS, and CORE IMPACT. These toolkits allow professional security consultants to reposition and deeply test networks in a highly effective manner.
Appendix A, contains definitive listings and details of tools and systems that can be used to easily assess services found.
Appendix B, lists good sources of publicly accessible vulnerability and exploit information so that vulnerability matrices can be devised to quickly identify areas of potential risk when assessing networks and hosts.
Appendix C, lists the exploit and auxiliary modules found in MSF, IMPACT, and CANVAS, along with GLEG and Argeniss add-on packs.
This book assumes you are familiar with IP and administering Unix-based operating systems, such as Linux or Solaris. A technical network administrator or security consultant should be comfortable with the contents of each chapter. To get the most out of this book, you should be familiar with:
The IP protocol suite, including TCP, UDP, and ICMP
Workings of popular Internet network services, including FTP, SMTP, and HTTP
At least one Unix-like operating system, such as Linux, or a BSD-derived platform like Mac OS X
Configuring and building Unix-based tools in your environment
Firewalls and network filtering models (DMZ segments, bastion hosts, etc.)
Mirror Site for Tools Mentioned in This Book
URLs for tools in this book are listed so that you can browse the latest files and papers on each respective site. If you are worried about Trojan horses or other malicious content within these executables, they have been virus-checked and are mirrored at the O’Reilly site http://examples.oreilly.com/networksa/tools/.
Using Code Examples
This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You don’t need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book doesn’t require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code doesn’t require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but don’t require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: "Network Security Assessment, Second Edition, by Chris McNab. Copyright 2008 Chris McNab, 978-0-596-51030-5.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at email@example.com.
Conventions Used in This Book
The following typographical conventions are used in this book:
Indicates example URLs, passwords, error messages, filenames, emphasis, and the first use of technical terms
Indicates commands, IP addresses, and Unix command-line examples
Constant width italic
Indicates replaceable text
Constant width bold
Indicates user input
This icon signifies a tip, suggestion, or general note.
This icon indicates a warning or caution.
Comments and Questions
Please address comments and questions concerning this book to the publisher:
|O’Reilly Media, Inc.|
|1005 Gravenstein Highway North|
|Sebastopol, CA 95472|
|800-998-9938 (in the United States or Canada)|
|707-829-0515 (international or local)|
There’s a web page for this book that lists errata, examples, and any additional information. You can access this page at:
To comment or ask technical questions about this book, send email to:
For more information about books, conferences, Resource Centers, and the O’Reilly Network, see the O’Reilly web site at:
As I look back over the last 27 years of my life, I realize that I have met a handful of key individuals to whom I owe a great deal, as I truly believe that I wouldn’t have ended up here without their input in one form or another: Wez Blampied, Emerson Tan, Jeff Fay, Bryan Self, Marc Maiffret, Firas Bushnaq, John McDonald, Geoff Donson, Kevin Chamberlain, Steve McMahon, Ryan Gibson, Nick Baskett, and James Tusini.
I am also extremely grateful for the positive support from the O’Reilly Media team since 2003, including Tatiana Apandi, Nathan Torkington, Jim Sumser, Laurie Petrycki, and Debby Russell.
The talented individuals I work alongside at Matta (http://www.trustmatta.com) deserve a mention, along with my colleagues at DarkStar Technologies. Without the support of the guys I work with, I would never get complex projects like this book finished on time!
Finally, many thanks to Glyn Geoghan for technical review of both editions of this book.
Guest Authors Featured in This Book
A big thanks to the following for ghostwriting and improving the following chapters of this book:
These individuals are recognized specialists in their respective areas and have made excellent contributions to this book. Without them, the book would not be such a comprehensive blueprint for security testing and assessment.