By Bryan Burns, Jennifer Stisa Granick, Steve Manzuik, Paul Guersch, Dave Killion, Nicolas Beauchesne, Eric Moret, Julien Sobrier, Michael Lynn, Eric Markham, Chris Iezzoni, Philippe Biondi
Book Price: $59.99 USD
£37.50 GBP
PDF Price: $48.99

Cover | Table of Contents | Colophon

Table of Contents

Chapter 1: Legal and Ethics Issues

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

In the summer of 2005, systems administrators and security researchers from all over the world gathered in Las Vegas, Nevada for Black Hat, one of the largest computer security conferences in the world. On the morning of the first day, Michael Lynn, one of the authors of this book, was scheduled to speak about vulnerabilities in Cisco routers. These vulnerabilities were serious: an attacker could take over the machines and force them to run whatever program the attacker wanted.

Cisco did not want Lynn to give the presentation. After last-minute negotiations with Lynn's employer, ISS, the companies agreed that Lynn would have to change his talk. A small battalion of legal interns converged on the convention floor the night before the speech and seized the CDs that contained Lynn's presentation slides for the talk and removed the printed materials out of the conference program.

Lynn, however, still wanted to give the original speech. He thought it was critical that system administrators know about the router flaw. A simple software upgrade could fix the problem, but few, if any, knew about the vulnerability. Lynn thought disclosure would make the Internet more secure. So, he quit his job at ISS and gave the talk he originally planned.

That evening, Cisco and ISS slapped Lynn, and the Black Hat conference, with a lawsuit.

We live in the Information Age, which means that information is money. There are more laws protecting information now than there were 25 years ago, and more information than ever before is protected by law. Cisco and ISS alleged that Lynn had violated several of these laws, infringing copyrights, disclosing trade secrets, and breaching his employment contract with ISS.

Lynn came to me because I've spent the last 10 years studying the law as it relates to computer security. I've advised coders, hackers, and researchers about staying out of trouble, and I've represented clients when trouble found them anyway. I've given speeches on computer trespass laws, vulnerability disclosure, and intellectual property protection at Black Hat, to the National Security Agency, at the Naval Postgraduate School, to the International Security Forum, and at Australia's Computer Emergency Response Team conference. I've been a criminal defense attorney for nine years and have taught full time at Stanford Law School for the last six years.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Core Issues

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

I began this exploration of security ethics and issues with Michael Lynn and the Black Hat affair, not because of its notoriety in security circles, and certainly not to embarrass or promote him or the companies that filed suit, but because the case really does raise fascinating legal issues that the security marketplace is going to see again and again. You can substitute one company's name for another, or one defendant for another, and the issues remain just as current. This chapter is going to review these legal issues in an open-minded way. Let's begin with a few simple items from the Lynn case.

One of the allegations was the misappropriation of trade secrets. A trade secret is information that:

(1) Derives independent economic value, actual or potential, from not being generally known to the public or to other persons who can obtain economic value from its disclosure or use; and (2) Is the subject of efforts that are reasonable under the circumstances to maintain its secrecy.

What was the secret? Lynn did not have access to Cisco source code. He had the binary code, which he decompiled. Decompiling publicly distributed code doesn't violate trade secret law.

Could the product flaw itself be a protected trade secret? In the past, attorneys for vendors with flawed products have argued that researchers would be violating trade secret law by disclosing the problems. For example, in 2003, the door access control company Blackboard claimed a trade secret violation and obtained a temporary restraining order preventing two researchers from disclosing security flaws in the company's locks at the Interz0ne II conference in Atlanta, Georgia. What if we had the same rule with cars? Imagine arguing that the fact that a car blows up if someone rear ends you is a protected secret, because the market value drops when the public knows the vehicle is dangerous. No thoughtful judge would accept this argument (but judges don't always think more clearly than zealous attorneys do).

Even if there is some kind of trade secret, did Lynn misappropriate it? Misappropriation means acquisition by improper means, or disclosure without consent by a person who used improper means to acquire the knowledge.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Computer Trespass Laws: No "Hacking" Allowed

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Perhaps the most important rule for penetration testers and security researchers to understand is the prohibition against computer trespass.

There are both common law rules and statutes that prohibit computer trespass under certain circumstances. (Common law rules are laws that have developed over time and are made by judges, while statutes are written rules enacted by legislatures—both types of laws are equally powerful.) There are also Federal (U.S.) statutes and statutes in all 50 U.S. states that prohibit gaining access to computers or computer networks without authorization or without permission.

Many people informally call this trespassing hacking into a computer. While hacking has come to mean breaking into computers, the term clouds the legal and ethical complexities of laws that govern use of computers. Some hacking is legal and valuable, some is illegal and destructive. For this reason, this chapter uses the terms computer trespass and trespasser or unauthorized access and attacker to demarcate the difference between legal and illegal hacking.

All statutes that prohibit computer trespass have two essential parts, both of which must be true for the user to have acted illegally. First, the user must access or use the computer. Second, the access or use must be without permission. The federal statute has an additional element of damage. Damage includes nonmonetary harm such as altering medical records or interfering with the operation of a computer system used for the administration of justice. Damage also includes causing loss aggregating at least $5,000 during any one-year period. In practice, plaintiffs do not have much trouble proving damage because most investigations of a computer intrusion will cost more than $5,000 in labor and time.

Some state statutes define criminal behavior, which means that the attacker can be charged with an offense by the government and, if found guilty, incarcerated. Some state statutes and the federal law define both a crime and a civil cause of action, for which the owner of the computer system could sue the attacker for money.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Reverse Engineering

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The human race has the ability and perhaps even the innate urge to study its environment, take it apart, and figure out how things work. One might argue it is why we are who we are. Reverse engineering is one expression of this tinkering impulse.

However, when you consider reverse engineering in the field of computers and software, the practice can conflict with legal rules designed to protect intellectual property. While intellectual property law generally recognizes reverse engineering as legitimate, there are some important exceptions that have ramifications for security engineers and professionals. There are three intellectual property rules that may affect your ability to legally reverse engineer: copyright law, trade secret law, and the anti-circumvention provisions of the Digital Millennium Copyright Act.

A fundamental technique used by security researchers is to take a "known product and working backward to divine the process which aided in its development or manufacture." The Ninth Circuit Court of Appeal has defined reverse engineering in the context of software engineering as:

(1) reading about the program;

(2) observing the program in operation by using it on a computer;

(3) performing a static examination of the individual computer instructions contained within the program; and

(4) performing a dynamic examination of the individual computer instructions as the program is being run on a computer.

So, many methods of reverse engineering pose no legal risk of copyright infringement. However, emulating, decompilation, and disassembly will require at least partial reproduction of the original code. And copyright law protects software. Copyright law grants to the copyright owner certain exclusive rights in the work, even when copies of the item are given away or sold. These rights include: the right to reproduce the work; the right to prepare derivative works; the right to distribute copies of the work; the right to perform the work publicly; and the right to display the work publicly. Thus, some reverse engineering will create infringing copies of a software program.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Vulnerability Reporting

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

One of the more vigorous public policy debates in the security field centers on publication of information about security vulnerabilities. Some argue that vulnerability publication should be restricted in order to limit the number of people with the knowledge and tools needed to attack computer systems. Restriction proponents are particularly concerned with information sufficient to enable others to breach security, especially including exploit or proof-of-concept code.

The benefits of publication restrictions theoretically include denying script kiddies attack tools, reducing the window of vulnerability before a patch is available, and managing public overreaction to a perception of widespread critical insecurity.

Opponents of publication restrictions argue that the public has a right to be aware of security risks, and that publication enables system administrator remediation while motivating vendors to patch. They also question whether restricting white hat researchers actually deprives black hats of tools needed to attack, under the theory that attackers are actively developing vulnerability information on par with legitimate researchers.

Today many, if not most, security researchers have voluntarily adopted a delayed publication policy. While these policies may differ in detail, they come under the rubric of responsible disclosure. The term has come to mean that there's disclosure but no distribution of proof-of-concept code until the vendor issues a patch. Once the patch is issued, it in itself can be reverse engineered to reveal the security problem, so there is little point in restricting publication after that time. In return, responsible vendors will work quickly to fix the problem and credit the researcher with the find.

Various businesses that buy and sell vulnerabilities are threatening this uneasy balance, as are researchers and vendors that refuse to comply. For example, in the month of January 2007, two researchers published daily flaws in Apple's operating system without giving advance notice of those flaws to the company.

Can we regulate security information? The dissemination of pure information is protected in the U.S. by the First Amendment. Many cases have recognized that source code, and even object code, are speech-protected by the First Amendment, and as a general principle, courts have been loath to impose civil or criminal liability for truthful speech even if it instructs on how to commit a crime. (The infrequent tendency of speech to encourage unlawful acts does not constitute justification for banning it.)

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

What to Do from Now On

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

I cannot cover this topic completely without devoting an entire book to it, and perhaps not even then. Security practices do not fit neatly into white hat or black hat categories. There are legal and ethical gray areas where most of you live and work. This book intends to give you technical skills in using an assortment of security tools, but it's how you use those tools that create the legal and ethical challenges with which this chapter, the legal system, and society grapple.

Any bozo can file a lawsuit, but you will usually receive some notification first, in the form of a demand or cease and desist letter. If you receive one of these, get advice from a lawyer. Perhaps the suit can be prevented or settled ahead of time.

Criminal charges often come without any advance notice to you. The FBI may show up at your door asking questions; they may have a warrant to seize your computers; they may ask permission to take your machines. You may never hear anything further from them, or you may get arrested months later. Local law enforcement investigates differently. If law enforcement comes to question you, ask for a lawyer immediately. You may have done nothing wrong, and you may want to cooperate, but that is something that a skilled attorney must help you with. Sometimes the police tell you that getting a lawyer is just making matters worse for you. Actually, it makes matters worse for them, because there's someone looking out for your interests and making sure that they keep their promises to you.

In less extreme situations, consider following the basic "What to do to protect yourself" bullet points throughout this chapter. They are certainly obvious, but you'd be surprised how seldom they are considered.

Ask for permission. Do not take things you are not intended to take. Do not break things. Publish your findings in open forums, using not-for-profit language and with good intent. Do not fake passwords. When you tinker with programs, make sure they are yours, do your research on your own time, on your own computers, without intent to gain financially or destroy something someone else has built.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 2: Network Scanning

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Virtually every network attack requires the IP address and port number of a vulnerable host in order to function. For example, if you have an Apache exploit ready to use, you need the IP address (and possibly the port number if the server is running on a nonstandard port) of a computer running Apache. Network scanners can provide you with this information, letting you know not only what IP addresses and ports are open but also what applications are running on which port.

Even if you don't have any specific intent, running a network scanner against a host or subnet provides valuable information you couldn't gather otherwise. Modern scanners can give you a feel for an entire network topology within a handful of seconds.

Scanners also are good at determining firewall rules and other access control policies. An administrator can verify his firewall is working properly using these techniques. Similarly, an attacker can use the same tricks to find holes in firewall coverage or simply learn the firewall rules to tailor his attack.

There are a number of network scanners out there, and each supports a different feature set and operates in a slightly different fashion. That said, most network scanners follow the same basic principles.

Networked applications communicate by sending packets back and forth. Scanners can determine whether an application is running on a computer by sending a packet that should elicit a response and waiting to see what comes back. If a response is sent, the application is likely to be running.

Some applications such as PortSentry exist for the sole purpose of confusing or frustrating port scans. Additionally, firewall features like SYN-cookies can make ports appear open when they are actually closed.

Most Internet applications communicate using either the TCP or UDP protocols. Both protocols use the concept of ports to allow for multiple applications to coexist on a single IP address. Both UDP and TCP support 65,536 (2¹⁶) distinct ports that applications can choose to bind to. Most applications have default ports that are used the vast majority of the time. HTTP (web) servers typically use TCP port 80. SMTP (email) servers almost always use TCP port 25. DNS servers use UDP port 53, and so on.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

How Scanners Work

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

There are a number of network scanners out there, and each supports a different feature set and operates in a slightly different fashion. That said, most network scanners follow the same basic principles.

Networked applications communicate by sending packets back and forth. Scanners can determine whether an application is running on a computer by sending a packet that should elicit a response and waiting to see what comes back. If a response is sent, the application is likely to be running.

Some applications such as PortSentry exist for the sole purpose of confusing or frustrating port scans. Additionally, firewall features like SYN-cookies can make ports appear open when they are actually closed.

Most Internet applications communicate using either the TCP or UDP protocols. Both protocols use the concept of ports to allow for multiple applications to coexist on a single IP address. Both UDP and TCP support 65,536 (2¹⁶) distinct ports that applications can choose to bind to. Most applications have default ports that are used the vast majority of the time. HTTP (web) servers typically use TCP port 80. SMTP (email) servers almost always use TCP port 25. DNS servers use UDP port 53, and so on.

The file /etc/services on most Unix machines contains a mapping of common applications to their default port number. For example, the following entry lets us know that IMAP servers typically use TCP port 143:

imap   143/tcp    # Internet Message Access Protocol

Network scanners determine what network applications are running on a given computer by testing TCP or UDP ports to see whether they are accepting connections. If TCP port 80 is open on a given IP address, it can be assumed that an HTTP server is running on that computer. Some scanners such as Scanrand (see the later section "") only tell you which ports are open, while others such as Nmap (see the later section "") or Unicornscan (see the section "," also later in this chapter) can communicate with the application to verify its guess or even identify the version of the application running.

A simple TCP scan of the computer at IP address 1.1.1.1 might involve attempting a connection to 1.1.1.1:1, then 1.1.1.1:2, then 1.1.1.1:3, and so on, until all ports have been attempted. (In reality, modern scanners are much more sophisticated about how they perform their scanning.)

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Superuser Privileges

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The network scanners discussed in this chapter all function by sending packets with very special parameters to the computer being scanned. Most Unix-like operating systems (such as Linux or Mac OS X) require superuser privileges in order to send these packets. Unicornscan and Nmap's connect scan (see ) mode work with normal user privileges, but advanced Nmap scans and Scanrand both require superuser privileges. Nmap works fine on Windows with an unprivileged user account.

Instead of logging in as root to gain superuser privileges, you can use sudo (see ) to temporarily elevate your privileges.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Three Network Scanners to Consider

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The following three network scanners are covered in this chapter. Here's a quick introduction to each of them and where to get them:

Nmap ()

Nmap is the oldest, most popular, and most feature-rich of the three scanners. First released in 1997, it has seen four major releases in the past decade. Nmap is widely available for most Unix platforms as well as Windows, and has both command-line and graphical interfaces. Nmap has been integrated into a number of commercial security products as well.

Unicornscan ()

While Unicornscan isn't quite as feature-rich as Nmap, it was designed with speed and scalability in mind. The packet-per-second rate can be precisely controlled to allow for very fast scans, or for slower scans so as to not exceed network constraints. Unicornscan also supports sophisticated UDP scans by speaking application protocols instead of sending empty scan packets. Precompiled packages are only available for a few operating systems; otherwise, it must be compiled from source code.

Scanrand ()

Scanrand is part of the Paketto Keiretsu toolkit by Dan Kaminsky. While it has the most limited feature set of the tools presented here, it is designed with one thing in mind: sheer speed. Scanrand uses a clever technique of encoding information in the headers of TCP SYN packets, allowing for very fast stateless scanning of a large set of addresses and ports. Scanrand and Paketto packages are available for most Unix operating systems.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Host Discovery

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

When presented with an unknown network, one of the first orders of business for scanning is to determine which IP addresses have computers listening on them. This is particularly important when exploring a network behind a Network Address Translation (NAT) device (see "Endpoint/Host" in ) where only a tiny percentage of available IP addresses may be in use. For example, on my home network, I have three class C networks defined (762 IP addresses), but 12 of those IP addresses are in use only, meaning that nearly 99 percent of the address space is unused. Host scans (also known as ping sweeps) quickly identify which IP addresses have computers attached and allow you to narrow the task at hand significantly.

Nmap provides the -sP option to perform a host scan. By default, Nmap sends both an ICMP echo request (also known as ping) packet as well as a TCP SYN packet to port 80 (the default web server port) to determine whether a computer is listening on a given IP address. If the IP addresses being scanned are on the same subnet as the scanner, ARP packets are used instead; it is a faster and more reliable way to see which IP addresses are in use. Here's an example of Nmap scanning the first 20 hosts of a subnet:

[bryan@nereid bryan] sudo nmap -n -sP 10.150.9.1-20

Host 10.150.9.15 appears to be up.
MAC Address: 00:0C:F1:D2:29:4C (Intel)
Host 10.150.9.16 appears to be up.
MAC Address: 00:0B:DB:27:40:47 (Dell ESG Pcba Test)
Nmap finished: 20 IP addresses (2 hosts up) scanned in 0.646 seconds

The -n flag instructs nmap to not do name lookups on the IP addresses it scans. This often makes the scan faster as reverse DNS lookups can take a long time to complete. The DNS requests can be somewhat noisy as well, so if you're trying to be subtle with your scan, -n is usually a good idea.

From the above output, you can see that of the first 20 IP addresses in the subnet, two are in use only. If the subnet scanned is local, Nmap is nice enough to look up the MAC addresses in its database to tell you who manufactured the network card.

Sending ping (ICMP echo request) packets used to be a reliable way to determine whether a computer was listening at a given IP address. These days, with firewalls becoming more widely deployed, ping packets are sometimes blocked by default. For example: the firewall that comes with Windows XP automatically blocks ping packets unless TCP port 445 is also allowed. In addition to sending a ping packet, Nmap also tries to connect to TCP port 80 as a fallback, but what if the host is blocking both pings and port 80? By default, Nmap considers the IP address to be vacant. In the following example, Nmap fails to find any hosts, despite there being a Windows XP machine at 10.150.10.253:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Port Scanning

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The purpose of network scanning is to identify which IP addresses have computers attached, and which applications are running on those computers. The previous section discussed how to find the computers, now let's focus on finding the open ports.

The scanners discussed in this chapter (Nmap, Unicornscan, and Scanrand) are all complex tools with many options (Nmap alone has nearly 80 distinct command-line flags) but port scanning is so central to each that without any command-line flags, they perform a port scan, the only necessary argument being the host(s) to scan. By default, all three scanners use a SYN scan (see ), which provides a good blend of speed and reliability. Depending on the tool, many other scan types may be available. These are covered in detail later in this chapter. Here is output from each scanner when run against my desktop computer without any arguments:

bryan@firemaw:˜$ sudo nmap 10.150.9.46

Interesting ports on 10.150.9.46:
(The 1667 ports scanned but not shown below are in state: filtered)
PORT     STATE  SERVICE
21/tcp   open   ftp
22/tcp   open   ssh
80/tcp   open   http
427/tcp  closed svrloc
443/tcp  closed https
3689/tcp open   rendezvous
8080/tcp open   http-proxy

bryan@firemaw:˜$ unicornscan 10.150.9.46
Open                         ftp[   21]         From 10.150.9.46        ttl 64
Open                         ssh[   22]         From 10.150.9.46        ttl 64
Open                        http[   80]         From 10.150.9.46        ttl 64
Open                    http-alt[ 8080]         From 10.150.9.46        ttl 64

bryan@firemaw:˜$ sudo scanrand 10.150.9.46
bryan@firemaw:˜$

You may have noticed that Scanrand returned zero results. This is because by default it doesn't do any bandwidth throttling and sends packets as fast as it can. This often leads to a significant number of packets being dropped by intermediate network devices or the end host. By throttling back the bandwidth with the -b flag (), results are produced.

An obvious question arises from the above output: why are the results from each tool different? Looking beyond the different in output formats, Nmap reported 5 open ports, Unicornscan reported 4, and Scanrand reported 0. Scanrand's lack of output was caused by a lack of bandwidth throttling, but why do Nmap and Unicornscan differ? The answer has to do with the default ports.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Specifying Custom Ports

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

A scanner that only allowed you to use the default ports would be severely limited, so all the scanners we discuss allow you to input arbitrary ports to be scanned on the command line.

Nmap allows you to pick custom ports with the -p ports option. The ports argument is a comma-separated list of ports or port ranges. For example:

sudo nmap -p 21-25,80,100-150 target

Nmap also provides the -F flag, which instructs Nmap to perform a "fast" scan by only looking for ports specified in the nmap-services file. This file comes with Nmap and contains around 1200 ports, which is a small decrease from the 1,600+ ports that Nmap scans by default.

Nmap provides its own services file, nmap-services, instead of relying on the /etc/services file provided by the host (see ). Depending on your environment, the nmap-services file may contain more or fewer entries than what is already on your system. For example, my Linux services file contains 279 TCP ports while Nmap's contains 1,246. However, My OS X machine has both beat with 4,065 entries.

You can mix UDP ports and TCP ports together in the ports list by typing T: in front of the TCP ports and U: in front of the UDP ports. For example, to scan TCP ports 21 through 25 and 80 and UDP ports 5000 through 6000, you would type:

sudo nmap -pT:21-25,80,U:5000-6000 target

Finally, Nmap assumes a port of 1 if the left side of a range is blank, and 65535 if the right side is blank. Therefore, -p-100 is equivalent to -p1-100, and -p100- is equivalent to -p100-65535.

The most concise way to specify that Nmap should scan all ports is to use -p-, which is equivalent to -p1-65535.

Unicornscan lets you specify custom ports by appending them to the address with a colon (:) character. As with Nmap, the ports specification can be a comma-separated list of individual ports or a range of ports. For example:

unicornscan target:21-25,80,100-150

If no custom ports are specified, Unicornscan scans its default set of 291 ports. This default set can also be selected by using the special character q (for "quick") in place of a port list.

Scanrand supports custom ports using the same syntax as Unicornscan, by appending the port list after the address with a "

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Specifying Targets to Scan

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

As mentioned earlier, the only argument that is required by any of our scanners is which host or hosts to scan. All other options assume reasonable default configurations, but there is no such thing as a default host.

There are three ways to specify the target(s) of a scan:

Single host

All three tools let you specify a single IP address or domain name to instruct the scanner to perform a scan of that single host—for example, 1.2.3.4 or www.somedomain.com.

Classless Inter-Domain Routing (CIDR) notation

CIDR notation lets you specify an IP address or domain name followed by a forward-slash (/) character and the number of bits in the subnet mask. For example, to scan the class C (256 addresses) network around 10.0.0.1, you would type 10.0.0.1/24. Class B (65535 addresses) networks are represented by /16, class A (16 million addresses) by /8, and so on.

The CIDR notation /0 denotes all possible IP addresses (there are over four billion of them). Unicornscan happily accepts 0/0 as a valid scan target and will commence to scan the entire Internet. You really shouldn't do this as it won't finish in your lifetime, and you'll likely annoy lots of people in the process. Nmap is polite enough to not accept /0 as a valid input, but it does accept /1, which, at two billion addresses, is nearly as bad.

IP address ranges

By far the most flexible way to specify scan targets is to use the IP address range notation. This style lets you enter comma-separated IP addresses and IP address ranges into each octet of the target. For example, to scan all (valid) IP addresses in the class C network around 10.0.0.1, you would type 10.0.0.1-254. This is roughly equivalent to the CIDR notation /24, but is slightly superior since 10.0.0.0 and 10.0.0.255 aren't valid IP addresses to scan, yet are included in the CIDR notation. The IP range notation allows you to express complex target lists that are impossible with CIDR notation, such as:

10.1,3,5,7,9.50-100,150-200.1-5,250-254

IP address ranges can come in handy for doing specialized scans of large networks. For example, the 10.0.0.0/8 network is commonly used on the inside of a NAT device (see ). In a large network this will likely be subdivided into smaller subnets, each with its own router. It is common practice for routers to be given an IP address at the beginning or end of a subnet range (for example, 10.5.5.1 or 10.5.5.254). To quickly scan the entire 10.0.0.0/8 network for routers that use BGP (a common router protocol that uses port 179), you could use:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Different Scan Types

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

By default, all three scanners perform a TCP SYN scan, but a variety of other scan types are available (with the exception of Scanrand, which has only one scan mode). SYN scans are the default because they are the most likely to successfully return useful results; however, depending on the network environment, other scan types can often return useful information missed by the default scan.

There are two types of UDP scans supported by our tools: empty packet scans and protocol data scans.

Empty packet scans involve sending UDP packets without any data to a port and waiting to see whether a result is returned. If an ICMP error such as "port unreachable" is returned, it can be assumed that the port is closed. If no response is seen, then the port is considered open or filtered. As mentioned in , this inability to differentiate between an open and firewalled port is a severe limitation of empty packet scans. Nmap is the only scanner under discussion that uses this technique. To instruct Nmap to perform an empty packet UDP scan, use the -sU flag. For example:

sudo nmap -sU target

Protocol data scans are a more sophisticated approach to UDP scanning that involves sending valid application protocol data in UDP packets to ports to see whether an application responds. Using this technique, ports are only assumed to be open if they respond to the protocol data packets with a nonerror response. Since this technique involves speaking to listening applications, it is more likely to be logged or even cause unexpected behavior such as crashing sensitive applications. Unicornscan supports this mode for UDP scans only, which makes it a good choice for accurate scanning of UDP ports. To perform a protocol data scan with Unicornscan, use the -mU flag, such as:

sudo unicornscan -mU target

Nmap also supports sending protocol data to UDP ports by using the application fingerprinting (-sV) functionality mentioned in Here's an example of Nmap performing a UDP scan with protocol data:

sudo nmap -sU -sV target

Mixing UDP and application fingerprinting scans in Nmap can lead to extremely slow scans. If possible, limit the ports to be scanned to the most interesting (see ).

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Tuning the Scan Speed

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

What is the ideal speed to perform a network scan? There are three good reasons why the answer is rarely "as fast as possible."

• Every network has a maximum capacity that when reached will cause packets to be silently dropped. Additionally, each host has finite processing and memory resources that can also cause packets to be dropped if received too fast. The end result is that if your scanner sends packets as fast as it possibly can, it is likely to cause packet loss, which leads to inaccurate results. (A scanner will likely interpret a dropped packet as being caused by a firewall or similar device.) We saw this behavior in when Scanrand was run without any throttling.
• If you are scanning a network other than your own, there is the possibility of an intrusion detection system (IDS) or intrusion prevention system (IPS) device watching your every move. These devices are often configured to detect scans, and perhaps take an action in response (such as block your IP address for a period of time). Due to the way scan detection works, the faster the scan, the more likely it is to be detected.
• Scans can wreak havoc on stateful network devices such as firewalls and NATing routers. Each packet of a scan typically represents a new connection, and a full-speed scan can easily exceed the resources of intermediary network devices. Depending on your network infrastructure, it is quite possible to perform a DoS (Denial of Service) attack on yourself by running a scan too fast. (I have personally crashed a number of commercial-grade firewalls by running Nmap with the -T5 option.) Another complication is that many firewall and IPS devices respond to a flood of SYN packets by enabling SYN cookies, which makes every port appear to be open.

Since controlling scan speed is so important, all three scanners provide various mechanisms to prevent packets from being sent too quickly. Each tool has taken its own approach to the problem, as detailed in the following sections.

Nmap provides a number of command-line options to fine-tune performance and packet timing. I almost never use these options myself, though, because Nmap provides premade

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Application Fingerprinting

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Knowing that a given port is open is valuable information, but even more valuable is knowing what exact application is running on that port. The -sV option instructs Nmap to test for application type and version for all ports found to be open. The following example shows Nmap fingerprinting the open ports on my OS X host:

bryan@firemaw:˜$ sudo nmap -n -sV 10.150.9.46

Interesting ports on 10.150.9.46:
(The 1667 ports scanned but not shown below are in state: filtered)
PORT     STATE  SERVICE     VERSION
21/tcp   open   ftp         tnftpd 20040810
22/tcp   open   ssh         OpenSSH 3.8.1p1 (protocol 1.99)
80/tcp   open   http        Apache httpd 1.3.33 ((Darwin) PHP/4.4.1)
427/tcp  closed svrloc
443/tcp  closed https
3689/tcp open   rendezvous  Apple iTunes 6.0.4 (on Mac OS X)
8080/tcp open   http-proxy?

From the output, you can see that Nmap was able to identify the application version for all but one port (8080). Nmap relies on a user-submitted database of application fingerprints in order to identify applications. In this case, the server running on port 8080 (CherryPy) was obscure enough that a fingerprint wasn't available. When Nmap is unable to identify a port, it provides data to be submitted to the insecure.org web site so future versions will be able to identify the application out of the box.

By default, Nmap skips certain ports and less likely payloads when performing fingerprinting. To force it to use all payloads on all ports, use the -allports and -version-all options.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Operating System Detection

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

One powerful feature that separates Nmap from the other scanners discussed here is the ability to determine the operating system (OS) of the target host while performing a scan. When this feature is selected, Nmap sends a few dozen specially crafted packets to open and closed ports (if they were found during the initial scan) and carefully analyzes the responses. By comparing the results with a database of hundreds of different operating systems, Nmap is often able to determine the target system, or at least provide a reasonable guess. If the target supports TCP timestamps, Nmap is often able to determine the uptime of the host. This can be useful to differentiate between desktop machines and servers, or to see how out of date the OS kernel might be. A host with an uptime of many months or years has likely missed a number of important operating system security updates and may be ripe for further attention.

In order to accurately determine the target operating system, Nmap typically needs at least one open and one closed port on the target. Sometimes Nmap can find a match just using one or the other, but having both is always preferable.

To enable OS detection, add the -O flag to the scan command line. The following flags can be used in conjunction with OS detection to augment the results:

-v

This flag increases Nmap's verbosity. When used with -O, Nmap performs a TCP Initial Sequence Number (ISN) and IP ID analysis. These metrics can be used to determine how susceptible the target is to various forms of traffic spoofing. Targets that are reported as having incremental IP ID sequence generation are good candidates for idle scans (see ).

—osscan-limit

This flag instructs Nmap to perform OS detection only on hosts with at least one open and one closed port, leading to more accurate results.

—fuzzy or —osscan-guess

This flag instructs Nmap to make guesses about potential target operating systems when an exact match cannot be found.

Depending on the OS being scanned and the state of ports found, the results of the OS detection can vary from very accurate, to broad, to no matches at all. Here are some results of an OS scan performed on my subnet using the following command:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Saving Nmap Output

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

By default, Nmap displays results of the scan to the terminal, but it is often preferable to save the results to a file for later inspection. This is particularly useful when scanning a large network as the scan output can span tens of pages. Some tools even take Nmap scan files as input, which is yet another reason to save the scan results to a file. Nmap can store the results of its scans in four different formats:

Normal

This is the same format as what is displayed to the terminal during a scan. The only difference is that the command-line options are printed at the top of the file as a reminder of what the scan was configured to do, and some runtime warnings are omitted.

Grepable

This format presents the results with one host per line in a concise fashion, meant to be easily processed with Unix text tools such as grep, sed, awk, and diff. Because of the condensed nature of this format, not all scan output may be preserved this way.

XML

This is the most powerful format, as the entire scan results are represented in highly structured XML for easy parsing by third-party applications. Unlike the Grepable format, all scan output is present in these files.

Script Kiddie

This format is presented solely as a joke and is simply the Normal output passed through a text-mangling filter.

These various output formats can be selected with the -otype filename option, where the type is N, G, X, or S. An additional option, -oA basename, is supported to simultaneously write the scan output in the Normal, Grepable, and XML formats. With this option, the files are named basename.nmap, basename.gnmap, and basename.xml. Multiple output formats can be specified using -o flags as well. For example, to write the output of a scan in normal and XML formats simultaneously, you would type:

sudo nmap -oN normal_output -oX xml_output target

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Resuming Nmap Scans

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

If Nmap scan results are being written to file in either Normal or Grepable format (see ), the scan can be resumed after interruption by using the —resume option. When resuming from a file, no command-line options are supported other than the file from which to resume (the original scan options are saved in the output file and are reused when the scan is resumed).

Only scans that span multiple hosts can be resumed, and the scan picks up after the last fully scanned host. Here's an example of a scan being interrupted after finishing one host and resuming to complete the scan on a second host:

bryan@firemaw:˜$ sudo nmap -oG grepable_output -n 10.150.9.15,143

Interesting ports on 10.150.9.15:
(The 1671 ports scanned but not shown below are in state: closed)
PORT    STATE SERVICE
22/tcp  open  ssh
111/tcp open  rpcbind
955/tcp open  unknown
^C
caught SIGINT signal, cleaning up
bryan@firemaw:˜$ sudo nmap —resume grepable_output

Interesting ports on 10.150.9.143:
(The 1672 ports scanned but not shown below are in state: filtered)
PORT    STATE SERVICE
139/tcp open  netbios-ssn
445/tcp open  microsoft-ds

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Avoiding Detection

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

As mentioned in , it is not uncommon for IDS or IPS devices to monitor your scan traffic. For various reasons, you may be interested in not being caught performing a network scan. One way to avoid detection is to slow the scan to a crawl (see ) in hopes of evading an IDS or IPS. While this works for most devices, the speeds necessary to avoid detection are so low that your scan can go from taking seconds to hours or even days. Nmap provides two alternate techniques you can use to avoid getting caught in the act. Ironically, neither technique prevents the scan from being seen, but rather they disguise your source address.

The first approach is to perform what is called an idle scan. With this technique, scan the target by spoofing packets from a zombie host and then bouncing packets off the zombie to see what ports are open on the target. This scan works only if the zombie uses predictable IP IDs and is not sending a large volume of network packets at the time of the scan. (See to determine whether a host has predictable IP IDs.)

To perform an idle scan, use the -sI zombie:port option. The zombie argument needs to be the address of a host with predictable IP IDs, and the port needs to be an open TCP port on that host (if no port is specified, Nmap tries port 80 by default).

It is a good idea to use -P0 (see ) with an idle scan so no packets are seen originating from your host. If you don't use this option, your host will send some initial host discovery packets prior to the spoofed scan, which could be used to trace the scan back to you.

Here's an example showing an idle scan of my desktop using port 3389 on 10.150.10.253 as a zombie:

bryan@firemaw:˜$ sudo nmap -P0 -sI 10.150.10.253:3389 10.150.9.46

Idlescan using zombie 10.150.10.253 (10.150.10.253:3389); Class: Incremental
Interesting ports on 10.150.9.46:
(The 1669 ports scanned but not shown below are in state: closed|filtered)
PORT     STATE SERVICE
21/tcp   open  ftp
22/tcp   open  ssh
80/tcp   open  http
3689/tcp open  rendezvous
8080/tcp open  http-proxy

From the perspective of the target (10.150.9.46), all packets from this scan came from 10.150.10.253, even though the host performing the scan has an IP address of 10.150.9.45. Even if an IDS or IPS had detected the scan, the host running Nmap will not be associated with the event.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Conclusion

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Network scanning provides a wealth of information about the target network, which is valuable regardless of whether you're trying to attack the network or protect it from attack. While performing a basic scan is a simple matter, the network scanners covered in this chapter provide a wide array of options to tweak your scan to achieve the best results. By taking advantage of these advanced features, you can make your scans more accurate, less likely to be detected, and faster to complete. If by this point you're still not sure which scanner is right for you, the answer is almost certainly Nmap. The other scanners have their own strengths, but Nmap's huge list of features and solid implementation make it the go-to scanner for most scans.

—Bryan Burns

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 3: Vulnerability Scanning

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Vulnerability scanning consists of looking for known vulnerabilities in known products. The traffic sent is very target-specific, as opposed to the traffic sent by the tools described , which require a lot of pseudorandom traffic.

A vulnerability scanner can execute intrusive or nonintrusive tests. An intrusive test tries to exercise the vulnerability, which can crash or alter the remote target. A non-intrusive test tries not to cause any harm to the target. The test usually consists of checking the remote service version, or checking whether the vulnerable options are enabled. Intrusive tests are typically much more accurate, but obviously they cannot be performed in a production environment. A nonintrusive test cannot determine for sure if a service installed is vulnerable, only if it might be vulnerable.

A vulnerability scanner such as Nessus (see ) differs from a penetration tool by the manner in which it exploits vulnerabilities. A scanner ensures that the vulnerability exists, but doesn't attempt to compromise the vulnerable software. A crash or degradation of the service is only a side effect of an intrusive test, not a goal.

I do not advise using any of the available vulnerability scanners to test an IDS. First, you can never be sure what type of test is performe