Secure Coding: Principles and Practices

Chapter 6. Automation and Testing

My test programs are intended to break the system, to push it to its extreme limits, to pile complication on complication, in ways that the system programmer never consciously anticipated. To prepare such test data, I get into the meanest, nastiest frame of mind that I can manage, and I write the cruelest code I can think of; then I turn around and embed that in even nastier constructions that are almost obscene.

—Donald Knuth, The Errors of T_EX, 1989

We frequently conduct seminars and briefings for software developers and security engineers from such industries as finance, communications, and pharmaceuticals. During each seminar, we ask the audience what software development processes and methodologies are in use at their particular companies. What do you think they tell us?

Without exception, the developers describe to us processes that, at best, can be called ad hoc. This discovery frightens us. If you want to build secure, reliable, and robust software, you simply can't do it without sticking to some kind of structured development methodology.

In previous chapters we examined the many ways that a program can come to contain security flaws throughout the various phases of its development and deployment lifecycle. In this chapter, we address the final step in putting that knowledge to practical use: testing and other means of automating parts of the development process. While you need to know what mistakes to avoid and what safe practices to ...