The alarming state of secure coding neglect

Only a quarter to a half of organizations do what their own programmers say is needed for the security of their code: automated code scans, peer security code reviews, and further code reviews by security experts. That’s one of the key findings in a survey of 430 professionals—mostly everyday programmers—carried out during December 2016 and January 2017 by O’Reilly Media along with the Software Improvement Group (SIG), an international consultancy firm specializing in software quality measurement and advice. ^[1]

Key findings include:

• Many organizations fail to perform desired security activities.
• External code reviews are considered important to improve security.
• Setting standards is more common than following through.

Code flaws can lead to serious security breaches

Let’s pause to consider the hostile environment in which code is running, and the dangers raised by insecure code:

• One of the central security protocols protecting the web—OpenSSL, the most popular software behind HTTPS—harbored for years a bug discovered only as late as 2014, called Heartbleed. No tool or test was able to find the bug.
• Sony Pictures suffered a major public relations embarrassment in 2014 when an attack shut down its computer systems and leaked both movies and email messages.
• A series of attacks on routers and consumer devices shows that the Internet of Things (IoT) is a dangerous ground for security breaches as well.

These examples were chosen because of the extensive news coverage they generated; hundreds of others could be added.

Although programmers ideally are trained to code securely, that by itself is insufficient to prevent security flaws. According to Rob van der Veer, a security consultant at SIG with experience as a computer scientist and CEO of various software firms, programmers are typically rewarded for adding features, which are highly visible to managers, sales teams, and customers. Anything programmers don’t make visible, for example, security, is less important.

With an idea of how important specialized security activities are, let’s now turn to the survey and illustrate how it informs our key findings.

External code reviews are seen as important to improve security

About 80% of respondents believed it important to show code to independent security experts, while another 16% said that it would be enough for developers to check each other’s code regularly. It’s notable that only 4% believe penetration testing, the most common kind of security assessment, to be sufficient. Penetration testing consists of creating common attacks, manually or through automated tools, to see whether an attacker can gain unauthorized access to the system. Although acknowledging its value, van der Veer cited several limitations of penetration testing:

• It can be performed only on a working system, so the feedback to developers comes very late, at a moment of high time pressure to release the software.
• It consists of black box testing, which is comparable to checking the security of a bank by rattling the doors and windows. Thus, penetration testing cannot determine such critical issues as whether there is adequate logging, whether there are internal flaws such as unnecessary privilege promotion, etc.
• The tester conducts the testing over a short period of time, such as a week, whereas malicious intruders have no time constraints. Additionally, given the limited time available to the testers, they check for only the most common and serious flaws.

Nearly 50% of respondents said independent expert reviews are worth their price, but their organizations lack the budget for it. A little more than 10% said the service was not worth the price, and a similar number said that suitable experts were unavailable. Fewer than 20% of respondents actually conduct such reviews.

Many organizations fail to perform desired security activities

Respondents were also asked whether various other security practices were in place. Choices included: 1) they were sufficiently in place, 2) they were somewhat in place, 3) they should have been in place but were not, and 4) they weren’t relevant. The details below tell the same basic story in every case—only a minority of respondents think the practice is sufficiently in place, and a large chunk always reports they want to perform the practice and cannot do it at all.

Static code analysis tooling

This was reported as being used by 25% of respondents. One-third of those who didn’t use it said it was too expensive. The rest of the non-users were fairly evenly divided among other explanations: tools were not available for their technology, were too hard to use, had too many false positives, or were not usable in Agile development.

Code reviews

Respondents were polled about several different types of reviews. We don’t know how much overlap there is between the types.

• Peers: 44% of the organizations used peer code review, while another 29% of respondents would like to do these reviews but do not.
• Internal security experts: 44% of the organizations used internal experts, and another 36% of respondents would like to do these reviews but do not.
• External security experts: 37% of the respondents would like to do these reviews but do not.

In all these cases, less than half the time are the code reviews considered sufficiently in place.

Time limitations were also identified by several respondents, who complained that start-ups could not devote extra time to security or that customers did not value security enough to tolerate extra time for it. While one could forgive small companies for a lack of resources—and perhaps for racing toward product release without leaving time for a security check—in this survey:

• More than 30% of respondents worked for institutions with more than 1,000 employees.
• 10% worked in institutions of 401-1,000 employees.
• 10% worked in institutions of 101-400 employees.

One would expect companies of these sizes to have room for security code reviews, if they were a priority.

Setting standards is more common than following through

Organizations do somewhat better in setting standards than in following through. For instance, respondents indicated that 69% of organizations have company-wide security requirements, while 60% impose guidelines for secure coding. These numbers are higher than those shown in the previous section for security practices—but still come nowhere near being universal. And as before, fewer than half of the respondents said that the standard-setting is done sufficiently.

Since this is the first time we’ve conducted this survey, we don’t don’t know whether organizations have evolved and might have improved their security practices. However, the low usage of security reviews and testing tells us that many more Sonys will generate lurid news stories before management realizes the importance of security.

Establishing a return on investment for security practices

Why is there such a large gap between the aspirations of programmers and their actual practices? The experience of Software Improvement Group consultants provides a possible answer: although programmers in the cubicles appreciate the importance of expert feedback on their code, top-level managers reject their requests. According to van der Veer, executives and boards of directors fail to provide the time or money needed to carry out inspections.

Each time a new security breach hits the news, one might expect other organizations to be “scared straight.” (This term comes from an old initiative to reduce criminal behavior among teenagers by making them tour jails—an initiative that also failed.) The costs of dealing with breaches, no matter how demoralizing, never seem to justify the extra time and money that good security requires. Additionally, although a security flaw is sometimes traceable to a single line of code—as in Apple’s famous “curly braces” bug—breaches are often a simultaneous failure on several levels of the software stack and its implementation. So each company may be able to shift blame onto other actors, and even the user.

Therefore, instituting secure coding practices seems to require a massive culture change over many years—similar to the culture changes that led to Agile programming, test-driven development, continuous integration, and related quality practices. And like those, security proponents will probably have to demonstrate improvements to the bottom line: less maintenance, improved customer satisfaction, or other measurable incentives to bring everyone on board.

Hopefully, the organizations that get the message early will thrive.

---

This post is a collaboration between SIG and O’Reilly. See our statement of editorial independence.