Shifting Left for Application Security

Chapter 4. Scenarios: Making the Cultural Shift

Moving toward cloud native containerized architectures takes time. Many enterprises still rely on legacy applications, infrastructure, and traditional waterfall development. This often means a “bookend” security model: defining security requirements at the beginning of development, performing penetration testing in production, and not much in between. Leadership may understand that this approach doesn’t scale, but moving to DevSecOps is complex, requiring changes in how different teams relate to each other.

Often, the enterprise starts by building static code-analysis and composition-analysis tools into the development process. This checks the box of building security into the pipeline but falls short of a holistic solution. The responsibility for protecting the code still falls squarely on the security team. They may provide feedback to the developers, but the relationship is not as collaborative as a true DevSecOps approach requires. Only when a true cultural shift makes security everyone’s responsibility is end-to-end accountability possible. The result is the ability to deploy a mix of tools, including traditional security tools, throughout the pipeline from design to runtime.

This last point is key: no matter how good your vulnerability detection is during the development process, you still need strong runtime security. Even with security built into the design, it’s impossible to catch a DoS attack early in the pipeline. Tools like WAFs, which can be inserted throughout the process, provide a last line of defense and a strong source of feedback to help teams collaborate better on security.

While the cultural shift requires cooperation by all teams, it’s often the security team that must make the first move. By becoming more transparent, security teams can overcome their reputation as gatekeepers. Managing threat modeling and security policies as code helps make their initiatives and tactics accessible to engineers, promoting collaboration. The engineers know how the application has been designed, but security has spent time observing how it performs in production. Automated tests, WAFs, and other tools can help validate that the performance of the application meets the design requirements, which in turn makes it easier to ensure that security is built in rather than bolted on. One key to a successful shift is to cultivate a few champions who can show others the importance of shifting security left.

Example: A Large Bank in Europe

Faced with nimble competition, a large bank in Europe needed to build new infrastructure quickly to move their many data centers and employees to a modern application architecture while maintaining compliance with a host of financial and other regulations. Though they had firewalls and other security measures in place, their security and development teams were only loosely connected. Testing new applications usually only meant that the software was installed on a virtual machine (VM) and an email sent to the security team asking them to test it. Teams within the company had their own budgets, didn’t talk to each other much, and didn’t get along.

Adopting automation was a necessity, but the question of how to do it created a great deal of internal resistance. Security didn’t have the budget for the tools. The application developers didn’t want to take on the burden of a CI/CD pipeline. The network team didn’t want the responsibility of owning central automation that affected all the different teams. No one wanted to take on the job of managing tools for everyone else. At the same time, no one wanted to lose people, power, or budget. The result was an internal cold war.

The executive team understood the value of end-to-end security and gave more power to the application developers, asking them to work side by side with security on building a modern application. The shift involved moving from VM-based architecture to orchestrated containers. As they built a true DevSecOps pipeline, they selected WAFs and other tools that they could control and operate with representational state transfer (REST) APIs, helping them automate security and infrastructure. The overall result was greater collaboration and transparency across the organization.

Example: A Bank in Australia

Around the world, banking is heavily regulated and highly competitive. Customers sometimes choose a bank based on how quickly it can respond to a loan application. This fast pace is driving a strong push to cloud native architecture to serve banks’ many customer-facing channels and applications. The biggest internal challenge is getting applications to production quickly. The days of architecting an application for months and testing only at the end of the process are long gone.

To reduce friction, teams are moving to the cloud, working to manage infrastructure and security as code so that releases can be automated. Facing such a shift, the security team at one prominent Australian bank realized quickly that a gatekeeping posture doesn’t work in the cloud. Unlike on-premises infrastructure where security has a great deal of control, the cloud makes it possible for teams to spin up whatever they need without the security team’s knowledge or involvement. In order to continue providing value to the organization, the security team understood that they needed to collaborate closely with other teams.

Catalyzed by the move to the cloud, the security team took an active role in managing WAFs and other tools in the application runtime environment, building a feedback pathway back into the pipeline to inform changes to the design and build processes. The security team helps manage WAF configurations and other policies as code, making releases easier. The result is better cooperation among teams, helping the bank compete more effectively in a fast-moving marketplace.

Example: An Energy Company

In many enterprises, the rift between application developers and other teams is the main obstacle to cultural change. A large energy company in Europe faced an additional schism. Owned in a public/private partnership, the company’s goals were often split. Those on the private side, holding the purse strings, were usually the decision makers. The public side, accountable to the citizenry, was shackled in regulations and bureaucracy. This made a move to DevSecOps especially challenging, because shifting security left involves such tight cooperation among different teams. Adding to the friction was the fact that the decision makers often weren’t technical, which sometimes made it difficult for them to see the value in technology options. In fact, at the beginning of the process, the company had not adopted VMs, running instead on bare metal servers.

The main impediment to adopting cloud native DevSecOps was inertia. Many people wanted to keep things as they were rather than changing things they felt were already working. The DevSecOps story was hard to sell inside the company to people who didn’t want to lose their power or budgets, didn’t understand the technology, and didn’t care about features such as REST APIs and automation. The company knew it had to adapt to the times, but bureaucracy and inertia kept them where they were, without a roadmap for change. Eventually, the company hit the limits of the existing tools and was forced to look at new solutions.

The company began to develop a list of requirements, including telemetry, customization, and programmability. Champions within the company had to work hard to help the decision makers understand the capabilities and limits of the technologies. Ultimately, what made the difference was having conversations with other companies who had adopted DevSecOps tools, including shifting WAFs and other tools left in the pipeline. In the end, the move to DevSecOps was not about getting security and other teams to work together. The solution was educating the decision makers about why end-to-end security is important and how to implement it.

Example: A Telecommunications Company

The telecommunications world is always moving. As bandwidth has become a commodity, telecommunications companies must stay competitive by branching out into add-on products without losing focus on supporting network infrastructure, service delivery, and other IT goals. The transition to 5G is pushing telecommunications companies to adapt and move faster, and driving their adoption of cloud native architecture.

The cultural change in a telecommunications company can sometimes happen faster than in a bank or an energy company, because the security team is smaller. A telecommunications company might have a security team of 30 people, whereas a bank’s security might be 20 times that size. With a small security team, the challenge shifts to educating the application developers, who might see security as an afterthought. A knowledgeable champion can help engineers understand the problems that result from releasing code without the right level of controls in place.

Building security into the automation pipeline as the company transitions its architecture to cloud native helps reduce friction. By adding runtime controls and commit-time code scans, the company can begin its journey to a mature DevSecOps pipeline. Continual education can help the application developers understand that end-to-end security is everyone’s responsibility.