Chapter 1. Introduction
For, usually and fitly, the presence of an introduction is held to imply that there is something of consequence and importance to be introduced.
Many of us test web applications on either a daily or regular basis. We may be following a script of interactions (“click here, type XYZ, click Submit, check for OK message…”) or we might be writing frameworks that invoke batteries of automated tests against our web applications. Most of us are somewhere in between. Regardless of how we test, we need to get security testing into what we’re doing. These days, testing web applications must include some consideration of how the application performs in the face of active misuse or abuse.
This chapter sets the stage for our activities and how we are laying out tools and techniques for you to use. Before we talk about testing web applications for security, we want to define a few terms. What applications are we talking about when we say “web applications”? What do they have in common and why can we write a book like this? What do we mean when we say “security”? How different are security tests from our regular tests, anyway?
What Is Security Testing?
It’s often straightforward to test our application’s functionality—we follow the paths through it that normal users should follow. When we aren’t sure what the expected behavior is, there’s usually some way to figure that out—ask someone, read a requirement, use our intuition. Negative testing follows somewhat naturally and directly from positive testing. We know that a bank “deposit” should not be negative; a password should not be a 1 megabyte JPEG picture; phone numbers should not contain letters. As we test our applications and we get positive, functional tests built, building the negative tests is the next logical step. But what of security?
Providing Evidence
In security testing, we consider the entire set of unacceptable inputs—infinity—and focus on the subset of those inputs that are likely to create a significant failure with respect to our software’s security requirements—still infinity. We need to establish what those security requirements are and decide what kinds of tests will provide evidence that those requirements are met. It’s not easy, but with logic and diligence we can provide useful evidence to the product’s owner.
We will provide evidence of security fulfillment in the same way that we provide evidence of functional fulfillment. We establish the inputs, determine the expected outcome, and then build and execute tests to exercise the system. In our experience with testers that are unfamiliar with security testing, the first and last steps are the hardest. Devising antisecurity inputs and testing the software are the hardest things to do. Most of the time, the expected outcome is pretty easy. If I ask the product manager “should someone be able to download the sensitive data if they are not logged in?” it’s usually easy for him to say no. The hard part of providing evidence, then, is inventing input that might create that situation and then determining whether or not it happened.
Fulfilling Requirements
The ANSI/IEEE Standard 729 on software engineering defines a requirement as a condition or capability needed by a user to solve a problem or achieve an objective or as a condition or capability that must be met or possessed by a system…to satisfy a contract, standard, specification, or other formally imposed document. All testers test to requirements when they have requirements available. Even when requirements are not available in the form of a document full of “the software shall...” statements, software testers tend to establish consensus on the correct behavior and then codify it in their tests in the form of expected results.
Security testing is like functional testing because it is just as dependent on that understanding of “what behavior do we want?” It is arguable that security testing is more dependent on requirements than functional testing simply because there is more to sift through in terms of potential inputs and outputs. Security behavior tends to be less well defined in the minds of the requirements-writers, because most software is not security software. The software has some other primary purpose, and security is a non-functional requirement that must be present. With that weaker focus on security, the requirements are frequently missing or incomplete.
What about this idea of sufficiently fulfilling requirements? Since security is an evolving journey and since security is not usually our primary function, we don’t always do something just because it is more secure. True software security is really about risk management. We make sure the software is secure enough for our business. Sometimes a security purist may suggest that the software is not secure enough. As long as it satisfies the business owners—when those owners are aware of the risks and fully understand what they are accepting—then the software is sufficiently secure. Security testing provides the evidence and awareness necessary for the business to make the informed decision of how much security risk to accept.
Security Testing Is More of the Same
Security is a journey, not a destination. We will never reach a point where we declare the software secure and our mission accomplished. When we are performing functional testing, we usually have expected, acceptable inputs that will produce known, expected results. In security we do not have the same finiteness governing our expectations.
Let’s imagine we’re testing a requirement like “the convertIntToRoman(int) function will return
valid Roman numeral strings for all positive integers up to MAXINT.” If
we were only doing functional testing, we would supply “5” and make sure
we got “V” back. Boundary-value testing would check things like maximum
integer values, 0, −1, and so on. We would check for proper exception
handling of “−5” as input and make sure we did not get “–V” as output,
but rather an appropriately defined error response. Finally, exception
testing would use equivalence classes to make sure the function doesn’t
return something like “III.IVII” when given 3.42 as input and handles
weird strings like “Fork” as input with appropriate error
handling.
Security testing, however, goes beyond this by understanding the problem domain and crafting malicious inputs. For example, a tricky input for a Roman numerals algorithm is one that consists of many 9s and 4s (e.g., 9494949494). Because it requires use of recursion or references to the previous Roman numeral, it can lead to deep stacks in the software or excessive memory use. This is more than a boundary condition. When we do security tests on top of functional tests, we add a lot of test cases. This means we have to do two things to make it manageable: narrow down our focus and automate.
Anyone familiar with systematic software testing understands the concepts of boundary values and equivalence class partitioning. Without straying too deep into standard testing literature, let’s refresh these two points, because much of our web security testing will follow this same model. If you are comfortable with these fundamental processes in testing, you will find it easy to draw on them to organize your security testing.
Boundary values
Boundary values take a given input and test very carefully around its acceptable boundaries. For example, if an input is supposed to allow integers that represent percentages, from zero to 100 inclusive, then we can produce the following boundary values: –1, 0, 1, 37, 99, 100, 101. To produce boundary cases, we focus on the two values at the top and bottom of the range (zero and 100). We use the boundary value itself, one less, and one more for each of the boundaries. For good measure, we pick something in the middle that should behave perfectly well. It’s a base case.
Equivalence classes
When we’re trying to develop negative values for testing, we know that the set of inputs that are unacceptable is an infinite set. Rather than try to test some huge set of inputs, we strategically sample them. We break the set of infinity into groups that have some commonality—equivalence classes—and then we pick a few representative sample values from each group.
Following the example from Boundary values, we need to choose a few classes of illegal input and try them out. We might choose classes like negative numbers, very large positive numbers, alphabetic strings, decimal numbers, and some significant special values like MAXINT. Typically we would pick a small number of values, say two, for each class and add it to our test input set.
Security classes
The seven boundary values in Boundary values and the two values each from approximately nine equivalence classes in Equivalence classes reduce the set of negative data test cases from infinity to 25. That’s a good start. Now we start adding in security test cases, based on common attacks and vulnerabilities. This is how security testing can become a straightforward, common part of everyday functional testing. We choose special boundary values that have security significance and special equivalence class values that have security significance, and we fold those into our test planning and test strategy process.
There are a few commonly recognized security input classes: SQL injection strings, cross-site scripting strings, and encoded versions of other classes (as discussed in Recipes 5.8 and 12.1 and Chapter 4, respectively). For example, you can Base 64- or URL-encode some attack strings in order to slip past input validation routines of some applications. Now, unlike the boundary values and other equivalence classes, these security classes are effectively infinite. So, again, we strategically sample to make it a manageable set. In the case of encoding we can choose three or four encodings. This triples or quadruples our test set, taking 25 values to 75 or 100. There are ways around that because typically the system either fails on an encoding, or succeeds. If the system fails when you URL-encode –1, it will probably fail when you URL-encode 101, too. Thus, you could probably choose to Base 64 encode some values, URL-encode others, HTML-encode others, and multiply-encode some others. This gives you coverage over the encodings without quadrupling your test case size. Perhaps it only doubles to 50 test cases.
Now the attack strings for SQL injection and cross-site scripting are up to you. You have to exercise some discretion and choose a reasonable subset that you can get done in the time you have available. If you are working in a part of your system that is easy to automate, you might do dozens of test cases in each class. If you are performing manual testing, you should probably acquire a long list of different attack strings, and try different ones each time you do your testing. That way, although you don’t get every string tested on every test run, you will eventually get through a lot of different cases.
What Are Web Applications?
Web applications come in a variety of shapes and sizes. They are written in all kinds of languages, they run on every kind of operating system, and they behave in every conceivable way. At the core of every web application is the fact that all of its functionality is communicated using HTTP, and its results are typically formatted in HTML. Inputs are communicated using GET, POST, and similar methods. Let’s explore each of these things in turn.
Our definition of a web application is simply any software that communicates using HTTP. This may sound like a broad definition, and it is. The techniques we are showing you in this book apply to any technology based on HTTP. Notice that a web server that serves up static web pages does not fit our bill. There is no software. If you go to the same URL, you will see the exact same output, and there is no software that executes as a result of making the request. To be a web application, some kind of business logic (script, program, macros, whatever) must execute. There must be some kind of potential variability in the output. Some decisions must be made. Otherwise, we’re not really testing software.
There are a few other classes of software that fit this description of “web application” that we will only touch on a little bit here. Web services generally, and then broad architectures that use those services in a service-oriented architecture (SOA), will only be touched on a little bit in this book. They are important, but are a broad class of applications worth their own book. There are also some specialized business-to-business (B2B) and electronic data interchange (EDI) standards that are built on HTTP. We will not venture into that domain, either. Suffice it to say that the techniques in this book are the basic foundation for testing those applications also, but that security tests that understand the problem domain (B2B, SOA, EDI) will be more valuable than generic web security tests.
Terminology
To be clear in what we say, here are a few definitions of terms that we are going to use. We try hard to stay within the industry accepted norms.
- Server
The computer system that listens for HTTP connections. Server software (like Apache and Microsoft’s IIS) usually runs on this system to handle those connections.
- Client
The computer or software that makes a connection to a server, requesting data. Client software is most often a web browser, but there are lots of other things that make requests. For example Adobe’s Flash player can make HTTP requests, as can Java applications, Adobe’s PDF Reader, and most software. If you have ever run a program and seen a message that said “There’s a new version of this software,” that usually means the software made an HTTP request to a server somewhere to find out if a new version is available. When thinking about testing, it is important to remember that web browsers are just one of many kinds of programs that make web requests.
- Request
The request encapsulates what the client wants to know. Requests consist of several things, all of which are defined here: a URL, parameters, and metadata in the form of headers.
- URL
A Universal Resource Locator (URL) is a special type of Universal Resource Identifier (URI). It indicates the location of something we are trying to manipulate via HTTP. URLs consist of a protocol (for our purposes we’ll only be looking at
httpandhttps). The protocol is followed by a standard token (://) that separates the protocol from the rest of the location. Then there is an optional user ID, optional colon, and optional password. Next comes the name of the server to contact. After the server’s name, there is a path to the resource on that server. There are optional parameters to that resource. Finally, it is possible to use a hash sign (#) to reference an internal fragment or anchor inside the body of the page. Example 1-1 shows a full URL using every possible option.Example 1-1. Basic URL using all optional fieldshttp://fred:wilma@www.example.com/private.asp?doc=3&part=4#footer
In Example 1-1 there is a user ID
fred, whose password iswilmabeing passed to the server atwww.example.com. That server is being asked to provide the resource/private.asp, and is passing a parameter nameddocwith a value of 3 and a parameterpartwith a value of 4, and then referencing an internal anchor or fragment namedfooter.- Parameter
A parameters are key-value pairs with an equals sign (=) between the key and the value. There can be many of them on the URL and they are separated by ampersands. They can be passed in the URL, as shown in Example 1-1, or in the body of the request, as shown later.
- Method
Every request to a server is one of several kinds of methods. The two most common, by far, are GET and POST. If you type a URL into your web browser and hit enter, or if you click a link, you’re issuing a GET request. Most of the time that you click a button on a form or do something relatively complex, like uploading an image, you’re making a POST request. The other methods (e.g., PROPFIND, OPTIONS, PUT, DELETE) are used primarily in a protocol called Distributed Authoring and Versioning (DAV). We won’t talk much about them.
Fundamentals of HTTP
There are ample resources defining and describing HTTP. Wikipedia’s article (http://en.wikipedia.org/wiki/HTTP) is a good primer. The official definition of the protocol is RFC 2616 (http://tools.ietf.org/html/rfc2616). For our purposes, we want to discuss a few key concepts that are important to our testing methods.
HTTP is client-server
As we clearly indicated in the terminology section, clients make requests, and servers respond. It cannot be any other way. It is not possible for a server to decide “that computer over there needs some data. I’ll connect to it and send the data.” Any time you see behavior that looks like the server is suddenly showing you some information (when you didn’t click on it or ask for it expicitly), that’s usually a little bit of smoke and mirrors on the part of the application’s developer. Clients like web browsers and Flash applets can be programmed to poll a server, making regular requests at intervals or at specific times. For you, the tester, it means that you can focus your testing on the client side of the system—emulating what the client does and evaluating the server’s response.
HTTP is stateless
The HTTP protocol itself does not have any notion of “state.” That is, one connection has no relationship to any other connection. If I click on a link now, and then I click on another link ten minutes later (or even one second later), the server has no concept that the same person made those two requests. Applications go through a lot of trouble to establish who is doing what. It is important for you to realize that the application itself is managing the session and determining that one connection is related to another. Nothing in HTTP makes that connection explicit.
What about my IP address? Doesn’t that make me unique and allow the server to figure out that all the connections from my IP address must be related? The answer is decidedly no. Think about the many households that have several computers, but one link to the Internet (e.g., a broadband cable link or DSL). That link gets only a single IP address, and a device in the network (a router of some kind) uses a trick called Network Address Translation (NAT) to hide how many computers are using that same IP address.
How about cookies? Do they track session and state? Yes, most of the time they do. In fact, because cookies are used so much to track session and state information, they become a focal point for a lot of testing. As you will see in Chapter 11, failures to track session and state correctly are the root cause of many security issues.
HTTP is simple text
We can look at the actual messages that pass over the wire (or the air) and see exactly what’s going on. It’s very easy to capture HTTP, and it’s very easy for humans to interpret it and understand it. Most importantly, because it is so simple, it is very easy to simulate HTTP requests. Regardless of whether the usual application is a web browser, Flash player, PDF reader, or something else, we can simulate those requests using any client we want. In fact, this whole book ultimately boils down to using non-traditional clients (testing tools) or traditional clients (web browsers) in non-traditional ways (using test plug-ins).
Web Application Fundamentals
Building Blocks
Web applications (following our definition of “software that uses HTTP”) come in all shapes and sizes. One might be a single server, using a really lightweight scripting language to send various kinds of reports to a user. Another might be a massive business-to-business (B2B) workflow system processing a million orders and invoices every hour. They can be everything in between. They all consist of the same sorts of moving parts, and they rearrange those parts in different ways to suit their needs.
The technology stack
In any web application we must consider a set of technologies that are typically described as a stack. At the lowest level, you have an operating system providing access to primitive operations like reading and writing files and network communications. Above that is some kind of server software that accepts HTTP connections, parses them, and determines how to respond. Above that is some amount of logic that really thinks about the input and ultimately determines the output. That top layer can be subdivided into many different, specialized layers.
Figure 1-1 shows an abstract notion of the technology stack, and then two specific instances: Windows and Unix.
There are several technologies at work in any web application, even though you may only be testing one or a handful of them. We describe each of them in an abstract way from the bottom up. By “bottom” we mean the lowest level of functionality—the most primitive and fundamental technology up to the top, most abstract technology.
- Network services
Although they are not typically implemented by your developers or your software, external network services can have a vital impact on your testing. These include load balancers, application firewalls, and various devices that route the packets over the network to your server. Consider the impact of an application firewall on tests for malicious behavior. If it filters out bad input, your testing may be futile because you’re testing the application firewall, not your software.
- Operating system
Most of us are familiar with the usual operating systems for web servers. They play an important role in things like connection time-outs, antivirus testing (as you’ll see in Chapter 8) and data storage (e.g., the filesystem). It’s important that we be able to distinguish behavior at this layer from behavior at other layers. It is easy to attribute mysterious behavior to an application failure, when really it is the operating system behaving in an unexpected way.
- HTTP server software
Some software must run in the operating system and listen for HTTP connections. This might be IIS, Apache, Jetty, Tomcat, or any number of other server packages. Again, like the operating system, its behavior can influence your software and sometimes be misunderstood. For example, your application can perform user ID and password checking, or you can configure your HTTP server software to perform that function. Knowing where that function is performed is important to interpreting the results of a user ID and password test case.
- Middleware
A very big and broad category, middleware can comprise just about any sort of software that is somewhere between the server and the business logic. Typical names here include various runtime environments (.NET and J2EE) as well as commercial products like WebLogic and WebSphere. The usual reason for incorporating middleware into a software’s design is functionality that is more sophisticated than the server software, upon which you can build your business logic.
Web Application Structures
One of the ways we can categorize web applications is by the number and kind of accessible interfaces they have. Very simple architectures have everything encapsulated in one or two components. Complex architectures have several components, and the most complicated of all have several multicomponent applications tied together.
A component is a little hard to define, but think of it as an encapsulated nugget of functionality. It can be considered a black box. It has inputs, it produces outputs. When you have a database, it makes an obvious component because its input is a SQL query, and its output is some data in response. As applications become more complex, they are frequently broken down into more specialized components, with each handling a separate bit of the logic. A good hint, though not a rule, for finding components is to look at physical systems. In large, sophisticated multicomponent systems, each component usually executes on its own physically separate computer system. Frequently components are separated logically in the network, also, with some components in more trusted network zones and other components in untrusted zones.
We will describe several architectures in terms of both the number of layers and what the components in those layers generally do.
Common components
The most common web applications are built on a Model-View-Controller (MVC) design. The purpose of this development paradigm is to separate the functions of input and output (the “View”) from the operations of the business requirements (the “Model”) integrated by the “Controller.” This permits separate development, testing, and maintenance of these aspects of the web application. When arranged in a web application, these components take on a few pretty common roles.
- Session or presentation layer
The session or presentation layer is mainly responsible for tracking the user and managing the user’s session. It also includes the decorations and graphics and interface logic. In the session and presentation component, there is some logic to issue, expire, and manage headers, cookies, and transmission security (typically SSL). It may also do presentation-layer jobs such as sending different visualizations to the user based on the detected web browser.
- Application layer
The application layer, when present as a distinct layer, contains the bulk of the business logic. The session component determines which HTTP connections belong to a given session. The application layer makes decisions regarding functionality and access control.
- Data layer
When you have a separate data layer, you have explicitly assigned the job of storing data to a separate component in the software. Most commonly this is a database of some sort. When the application needs to store or retrieve data, it uses the data component.
Given the many components that are possible, the number of separate layers that are present in the system influence its complexity a great deal. They also serve as focal points or interfaces for testing. You must make sure you test each component and know what sorts of tests make sense at each layer.
One-layer web applications
An application that has a single layer puts all its business logic, data, and other resources in the same place. There is no explicit separation of duties between, say, handling the HTTP connection itself, session management, data management, and enforcing the business rules. An example one-layer application would be a simple Java server page (JSP) or servlet that takes a few parameters as input and chooses to offer different files for download as a result.
Imagine an application that simply stores thousands of files,
each containing the current weather report for a given zip code. When
the user enters their zip code, the application displays the
corresponding file. There is logic to test (what if the user enters
xyz as her zip code?) and there
are even security tests possible (what if the user enters /etc/passwd as her zip code?). There is
only the one logic (e.g., the one servlet) to consider, though.
Finding an error means you look in just the one place. Since we are
supposing that session tracking is performed right within the same
logic, and we are not using any special data storage (just files that
are stored on the web server), there is no session or data layer in
this example.
How do you test a one-layer web app? You have to identify its inputs and its outputs, as you would with any application, and perform your usual testing of positive, negative, and security values. This will contrast considerably with what you do in multilayer applications.
Two-layer web applications
As an application’s needs expand, a second component offloads some of the work to a separate process or system. Most commonly, if there are only two layers, there is usually a single session/application component and a data component. Adding a database or sophisticated data storage mechanism is usually one of the first optimizations developers make to an application whose needs are expanding.
A common abbreviation in describing web applications is LAMP, standing for Linux, Apache, MySQL, and PHP. There are many applications built on this paradigm, and most are two-layer applications. Apache and PHP collaborate to provide a combined session/application component, and MySQL provides a separate data component. Linux is not important for our purposes. It is mentioned here because it is part of the abbreviation. Any operating system can host the Apache, MySQL, and PHP components. This allows expansion, replication, and redundancy because multiple independent systems can provide session and application logic while a different set of individual machines can provide MySQL data services.
Good examples of two-layer applications include any number of blogging, content-management, and website hosting packages. The Apache/PHP software controls the application, while the MySQL database stores things like blog entries, file metadata, or website content. Access control and application functions are implemented in PHP code. The use of a MySQL database allows it to easily deliver features like searching content, indexing content, and efficiently replicating it to multiple data stores.
Knowing that you have a two-layer application means that you have to consider tests across the boundary between the layers. If your presentation/app layer is making SQL queries to a data layer, then you need to consider tests that address the data layer directly. What can you find out about the data layer, the relationships in the data, and the way the application uses data? You will want to test for ways that the application can scramble the data, and ways that bad data can confuse the application.
Three-layer web applications
When developers decide to divide their work into three or more layers, they have a lot of choices about which components they choose. Most applications that are complex enough to have three components tend to use heavyweight frameworks like J2EE and .NET. JSPs can serve as the session layer, while servlets implement the application layer. Finally, an additional data storage component, like an Oracle or SQL Server database implements the data layer.
When you have several layers, you have several autonomous application programming interfaces (APIs) that you can test. For example, if the presentation layer handles sessions, you will want to see whether the application layer can be tricked into executing instructions for one session when it masquerades as another.
The effect of layers on testing
Knowing the relationships between the components in your application makes an important difference to your testing. The application is only going to fulfill its mission when all the components are working correctly. You already have several ways that you can examine your tests to evaluate their effectiveness. Test coverage, for example, is measured in a variety of ways: how many lines of code are covered by tests? How many requirements are covered by tests? How many known error conditions can we produce? Now that you understand the presence and function of architectural components, you can consider how many components of the application are tested.
The more information you, as a tester, can provide to a developer about the root cause or location of an error, the faster and more correctly the error can be fixed. Knowing that an error, for example, is in the session layer or data layer goes a long way towards pointing the developer in the right direction to solve it. When the inevitable pressure comes to reduce the number of tests executed to verify a patch or change, you can factor in the architecture when making the decision on which tests are most important to execute. Did they make modifications to the data schema? Try to organize your tests around data-focused tests and focus on that component. Did they modify how sessions are handled? Identify your session management tests and do those first.
Web App Security Testing
Let’s bring all these concepts together now. With functional testing, we are trying to provide evidence to our managers, business people, and customers that the software performs as advertised. With our security testing, we are trying to assure everyone that it continues to behave as advertised even in the face of adverse input. We are trying to simulate real attacks and real vulnerabilities and yet fit those simulations into the finite world of our test plan.
Web security testing, then, is using a variety of tools, both manual and automatic, to simulate and stimulate the activities of our web application. We will get malicious inputs like cross-site scripting attacks and use both manual and scripted methods to submit them to our web application. We will use malicious SQL inputs in the same way, and submit them also. Among our boundary values we’ll consider things like predictable randomness and sequentially assigned identifiers to make sure that common attacks using those values are thwarted.
It is our goal to produce repeatable, consistent tests that fit into our overall testing scheme, but that address the security side of web applications. When someone asks whether our application has been tested for security, we will be able to confidently say yes and point to specific test results to back up our claim.
It’s About the How
There are lots of books out there that try to tell you why to perform security tests, when to test, or what data to use in your tests. This book arms you with tools for doing that testing. Assuming you’ve decided why you should test, it’s now time to test, and you have some test data, we will show you how to put all that together into a successful security test for your web application.
No discussion of security testing would be complete without considering automation, and that is what many of the tools in this book specifically promote. Each chapter will describe specific test cases and highlight automation possibilities and techniques.
How, Not Why
Every year millions of dollars (and euros, pounds, yen, and rupees) are spent developing, testing, defending, and fixing web applications that have security weaknesses. Security experts have been warning about the impact of software failure for a long time. Organizations are now coming to recognize the value of security in the software development lifecycle. Different organizations react differently to the need for security, however, and no two organizations are the same.
We are not going to tell you much about why you should include security testing in your testing methodology. There are ample books trying to address that question. We can’t cover what it means to your organization if you have poor security in your software or how you perform a risk analysis to determine your exposure to software-induced business risk. Those are important concepts, but they’re beyond the scope of this book.
How, Not What
We are not going to provide you with a database of test data. For example, we will tell you how you can test for SQL injection or cross-site scripting, but we won’t provide a comprehensive set of malicious inputs that you can use. There are plenty of resources for that sort of thing online and we’ll refer you to a few. Given the rapidly changing nature of software security, you’re better off getting up-to-the-minute attack data online, anyway. The techniques presented in these recipes, however, will last a long time and will be helpful in delivering attacks of many kinds.
How, Not Where
This book does not present a methodology for assessing your application looking for weak spots. Assessing a web application—once or on a continuing basis—is not what we’re helping you do. Assessors come in and find problems. They do not bring the deep, internal knowledge of the application that the QA staff and developers have. External consultants do not fit into the software development lifecycle and apply tests at the unit, integration, and system level. If you need an overall methodology on how to assess a web application from the ground up, there are many good books on how to do that. When it’s time to do some of the tasks mentioned in those books, though, you’ll discover that many are laid out in good detail within the recipes in this book.
How, Not Who
Every organization will have to decide who will perform security testing. It might be (and probably should be) a combination of both developers and testers. It can involve folks from the IT security side of the house, too, but don’t let them own security testing completely. They don’t understand software and software development. If security testing falls exclusively to the testing and quality side of the organization, then you will want someone with some software development skills. Although we are not developing a software product here, the scripts and test cases will be easier to use and reuse if you have experience with programming and scripting. Even operations staff might benefit from the recipes in this book.
How you decide whom to assign to these tasks, how you organize their work, and how you manage the security testing is beyond the scope of this book.
How, Not When
Integrating security testing, like any other kind of specialized testing (performance, fault tolerance, etc.), requires some accommodations in your development lifecycle. There will be additional smoke tests, unit tests, regression tests, and so on. Ideally these tests are mapped back to security requirements, which is yet one more place your lifecycle needs to change a little. We are going to give you the building blocks to make good security tests, but we won’t answer questions about what part of your test cycle or development methodology to put those tests into. It is difficult to develop security test cases when security requirements are not specified, but that is a topic for another book. Instead, we are going to help you build the infrastructure for the test cases. You will have to determine (by experimenting or by changing your methodology) where you want to insert them into your lifecycle.
Software Security, Not IT Security
If you play word association games with IT people and say “security,” they’ll often respond with “firewall.” While firewalls and other network perimeter protections play an important role in overall security, they are not the subject of this book. We are talking about software—source code, business logic—written by you, operated by you, or at least tested by you. We don’t really consider the role of firewalls, routers, or IT security software like antivirus, antispam, email security products, and so on.
The tests you build using the recipes in this book will help you find flaws in the source code itself—flaws in how it executes its business functions. This is handy when you need to check the security of a web application but you do not have the source code for it (e.g., a third-party application). The techniques are especially powerful when you have the source itself. Creating narrow, well-defined security tests allows you to facilitate root cause analysis right down to the lines of code that cause the problem.
Although there are products that call themselves “application firewalls” and claim to defend your application by interposing between your users and youro application, we will ignore such products and such claims. Our assumption is that the business logic must be right and that it is our job—as developers, quality assurance personnel, and software testers—to systematically assess and report on that correctness.
Get Web Security Testing Cookbook now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
