How companies adopt and apply cloud native infrastructure
Survey results reveal the path organizations face as they integrate cloud native infrastructure and harness the full power of the cloud.
Survey results reveal the path organizations face as they integrate cloud native infrastructure and harness the full power of the cloud.
Driven by the need for agility, scaling, and resiliency, organizations have spent more than a decade moving from “trying out the cloud” to a deeper, more sustained commitment to the cloud, including adopting cloud native infrastructure. This shift is an important part of a trend we call the Next Architecture, with organizations embracing the combination of cloud, containers, orchestration, and microservices to meet customer expectations for availability, features, and performance.
To learn more about the motivations and challenges companies face adopting cloud native infrastructure, we conducted a survey of 590 practitioners, managers, and CxOs from across the globe.
Key findings from the survey include:
In our analysis, we assigned experience levels to our respondents for some of the survey questions. New respondents work at organizations that have been cloud native for less than one year; early respondents’ organizations have been cloud native for one to three years; and sophisticated respondents work at organizations that have been cloud native for more than three years.
The results in Figure 1 aren’t surprising, given that more developers are making technology decisions. If you combine software practitioners (engineers, developers, and admins) with leads/architects, that’s nearly 70% of respondents with technical roles. We see cloud native creating an increasing need for technical and architectural leadership, and more overlap between lead roles and engineering work. We expect those in technical roles to exert more influence over tool selection and development in the months and years ahead.
Around half of respondents have been in their current position for three years or less (Figure 2). This points to the shifting nature of jobs as the industry responds to new technologies and workflows.
Cloud native has played a role in this shift. Important cloud native tools like Docker (released in 2013) and Kubernetes (released in 2015) are both relatively new. Containers, like Docker, and orchestration tools, like Kubernetes, are essential for creating the runway organizations need to transform their digital properties from monolith to microservice architectures—a task many companies are still in the early days of performing. With 72% of respondents having adopted cloud native in the last three years (see Figure 9, below), we see the tools and infrastructure changing quickly, and people learning new skills on the job.
Survey respondents whose organizations adopted cloud native three-plus years ago—referred to in this analysis as “sophisticated”—typically work at software companies (Figure 3). Respondents in the finance and banking industry show the opposite pattern, with a larger share of those new to cloud native and a smaller share of sophisticated respondents. A legacy of outdated back office applications and regulatory and security concerns makes the migration to cloud native for those in finance that much more difficult. However, competitive pressure from fintech startups and increased competition from payment providers and pay services like Apple, Google, and Amazon create the imperative to transition to a cloud native infrastructure.
Media and entertainment companies show a notable share of experienced cloud native respondents, with nearly all falling into the early adopter or sophisticated categories. This isn’t too surprising, given the essential role microservices play in that industry. Netflix is a good media company case study, with its early adoption of microservices, its contributions to chaos engineering, and its open source, multi-cloud continuous delivery platform Spinnaker (which Netflix and Google recently donated to the newly launched Continuous Delivery Foundation (CDF)). Cloud native adoption will likely increase in the media and entertainment space, as the distributed systems that make cord-cutting possible require cloud native infrastructures.
When asking about the types of cloud platforms respondents use (Figure 4), we define public cloud as services offered solely through a third-party provider; hybrid cloud as using a mix of private and third-party cloud services; and private cloud as a secure, on-premises cloud infrastructure with restricted access.
More than 40% of respondents indicate their organizations use a hybrid cloud infrastructure (Figure 4). We expect hybrid to remain a popular option for accommodating data that can’t be stored on a public cloud and for serving as an interim architecture for organizations progressively transitioning their services and applications to a cloud native architecture
A close read of the responses in Figure 5 shows the more sophisticated the respondent’s cloud implementation, the more likely they are to use public cloud providers. Those new to the cloud are more likely to use hybrid cloud options. A smaller share of respondents report using only private cloud. This could be reflective of industries that can’t put their data on a public cloud due to laws and regulations (finance and government, for example) and the complexity of migrating legacy systems.
Amazon AWS leads across experience levels (Figure 6), but we also see some interesting cloud provider combinations among respondents (Figure 7):
With 48% of respondents having adopted a multi-cloud architecture (Figure 7), we see a competitive marketplace, with organizations sampling from a mix of vendors. We suspect organizations are “kicking the tires” on cloud providers, looking to test proprietary features or trying to lower costs by using more than one vendor. And, the strategy helps keep vendors honest, as containers allow an easy switch between platforms. Most organizations also want to maximize performance and resiliency, and a multi-cloud strategy allows for scaling and redundancy to help mitigate the risk of costly downtime and potential data loss in the case of an incident.
While 68% of respondents said their organizations have adopted, or at least have begun to adopt, cloud native infrastructure, more than 30% of self-selecting respondents to a cloud native survey let us know they haven’t adopted cloud native (Figure 8). The results show both the great interest in cloud native, and the great potential—one of the reasons we expect cloud native to be of interest for many years to come. The cloud native ecosystem is not yet settled, and computing is not quite a utility, so we anticipate much change as trends around cloud native coalesce.
The charts that follow show results from the 68% of survey respondents whose organizations have adopted cloud native infrastructure (Figure 8).
The results in Figure 9 reflect the newness of the cloud native space, with 72% of respondents adopting cloud native in the last three years.
New attention to cloud native mirrors what we’ve seen on the O’Reilly online learning platform. Cloud native topics are among the fastest growing areas. In search and usage on the O’Reilly platform, Kubernetes is the fastest growing large topic, with the major cloud vendors and other cloud native tools all growing strongly as well. While the cloud has been around for more than 10 years, the acceleration in interest we see on the platform shows the cloud an ascendent topic with strong legs and a good foundation for sustained growth. In particular, patterns around cloud native components helped drive support for the Next Architecture, as we note in this report.
The levels of success noted by sophisticated adopters reflect how experience with cloud native infrastructure pays off (Figure 10). Sophisticated respondents had by far the largest share of extreme success with their cloud native adoption. More than 90% of sophisticated respondents rated their cloud native implementations as mostly successful or better, and no sophisticated respondents felt their implementations were unsuccessful.
Adopting a cloud native infrastructure is both complex and difficult, which is made clear from the survey results showing at least 40% of respondents citing challenges with finding skilled engineers, migrating from legacy architecture, responding to security and compliance demands, managing technical infrastructure, and transforming their corporate culture (Figure 11).
That lack of skills is a much bigger factor than hiring shows respondents are trying to adopt cloud native on their own—a sign that organizations are fundamentally structuring themselves around cloud native architectures, not looking to hire that skill from outside. We see in Figure 12 that early respondents struggle significantly more with lack of skills and company culture than the other categories, suggesting these are issues organizations should consider tackling first when adopting cloud native.
In that same vein, as organizations restructure internal processes and pipelines to adopt a cloud native architecture, they struggle to put a corporate culture in place that supports this new way of working. Cloud native adoption isn’t just about using the right tools or having the right technical infrastructure (though, those are important). Adopting a DevOps workflow that breaks down the barriers between teams and embraces a culture of collaboration is essential to implementing a successful cloud native architecture that relies on continuous integration, delivery, and improvement.
And finally, security and compliance hurdles are not going away, even if they are less prevalent than a few years ago, when health data and financial regulations limited what many organizations could even consider placing in the cloud. While some of those hurdles have been resolved, our respondents tell us that security and compliance continue to require attention when considering cloud native implementations.
DevOps was by far the top choice for who manages the respondents’ cloud native infrastructures (Figure 13). This is evidence that adopting a DevOps culture is critical to meeting the market demands—agility, speed to market, scaling, and reliability—that cloud native also addresses.
It’s telling, though, when looking at the responses in the context of maturity, the more sophisticated cohort said they depend on site reliability engineering (SRE) to manage their cloud native infrastructure (Figure 14). SRE is a practice started at Google where engineers take on both development and operations responsibilities to release software faster and more reliably. SREs split their time between developing and maintaining infrastructure by automating repetitive tasks (toil) to allow time to build new features. While DevOps is a set of principles that loosely define how teams should work together to remove silos and collaborate effectively, SRE serves as an implementation of DevOps and is a job role that will see increased demand as organizations continue their transitions to cloud native.
It’s no surprise that containers show as the most popular tool (Figure 15), as containers serve a key cloud native infrastructure role, improving software developer productivity, deployment speed and flexibility, platform independence, and enabling effective scaling. In addition, the high number of responses for orchestration and management underscores the strength of Kubernetes in this field. Orchestration and containers, combined with the cloud and microservices, form the technical foundation of the Next Architecture.
The percentages around service mesh and serverless were lower, likely because these technologies are relatively immature (Figure 15). However, it’s worth briefly outlining how they work and why they’re poised for adoption because we expect both tools to play large future roles in the cloud native space.
A service mesh is a configurable, low-latency infrastructure layer designed to ease the complexity of networking microservices and managing communication between them in large, distributed networks. Istio, arguably the most popular service mesh, was developed by Google, Lyft, and IBM as an open source solution, and only reached version 1.0 in July 2018. We’ll see more organizations turning to service mesh infrastructure as the tools mature and expanding organizations seek solutions for scaling their systems and managing the increasing complexity that comes with a global, distributed network.
While service mesh provides networking infrastructure, serverless provides another layer of abstraction for cloud developers. Despite the name, serverless does not mean there are no servers. Serverless architecture simply puts the onus of managing backend infrastructure on the cloud provider so developers can focus on building applications rather than the software that powers them. Serverless architecture is generally stateless, provisions resources on demand, and only incurs cost for the resources actually used, potentially enabling organizations to scale rapidly while saving money. However, the technology still has significant drawbacks, as outlined in a recent study by the University of California at Berkeley, including inadequate storage, and performance and security concerns. But you can expect to see these issues resolved within the next decade. As serverless matures, Berkeley researchers predict it will “become the default computing paradigm for the Cloud Era, largely replacing serverful computing and thereby bringing an end to the Cloud-Server Era.”
The survey results also reveal a story around cloud native experience and tool adoption (Figure 16). Sophisticated cloud native organizations are in a position to iteratively improve and move into tools like service mesh and serverless. Organizations new to cloud native may need to wait and learn before they can fully harness some of these tools.
The charts in this section show results from the 32% of survey respondents whose organizations have not adopted cloud native infrastructure (see Figure 8, above).
The interesting story here is that the same challenges were identified whether respondents’ organizations have or have not moved to cloud native (compare Figure 11 and Figure 17). Lack of skills, company culture, microservices migration, and security and compliance were shared challenges among adopters and non-adopters alike.
In Figure 18, it’s surprising to see 27% of respondents work at organizations with no plans for cloud native adoption. While we can’t determine exactly why they have no plans, it seems prudent to investigate cloud native even if the adoption timeline for your organization will be long. This result may also reflect the newness of the space, as some organizations might not yet realize how important cloud native is and will be.
When evaluating responses from the sophisticated cohort, a few lessons emerge for organizations considering cloud native and those that are in the early stages of cloud native implementations:
More generally, as companies rise to meet the increasingly real-time demands of users and customers, and the need to respond to nimble competitors, we expect organizations to find the cloud native approach a necessity. Cloud native makes it possible for companies to deploy features faster, more affordably, more reliably, and with less risk. As the cloud native market develops, we see many opportunities for tools and training to help ease the transition to new architectures and to bridge the cloud native skills gap.