The Shellcoder's Handbook: Discovering and Exploiting Security Holes, Second Edition

Chapter 20. Tracing for Vulnerabilities

The process of discovering vulnerabilities can be time consuming and extremely tedious. We can save time and increase our efficiency by developing and maintaining a toolkit specifically designed to discover flaws in targeted software packages. This toolkit should consist of utilities and technologies that will allow us to audit an application's source code and its compiled machine code. We should also include tools that allow us to audit an application while it is operating. This category of tools includes aggressive auditing technologies (such as fuzzers; see Chapter 17), as well as miscellaneous passive monitoring tools. Each of our tools allows us to examine the security of an application from a different perspective. The technology within each of our tools has its benefits as well as its weaknesses. By combining several of these technologies, we can eliminate many of their weaknesses while retaining their individual strengths.

In the second quarter of 2001, a project was begun to combine several technologies into one auditing solution, EVE. Each technology had its own weaknesses when used alone; for example, machine-code auditing was very effective in identifying single instances of potential security holes, but unfortunately, the task of determining whether the potential flaw could actually be exploited was extremely difficult if the application was not running. By building a machine-code auditing solution capable of auditing applications ...