Chapter 5. From DevOps to DataOps

Why It’s Time to Embrace “DataOps” as a New Discipline

Over the past 10 years, the technology industry has experienced the emergence of “DevOps.” This new set of practices and tools have improved the velocity, quality, predictability, and scale of software engineering and deployment. Starting at the large Internet companies, the trend toward DevOps is now transforming the way that systems are developed and managed inside the enterprise—often dovetailing with enterprise cloud adoption initiatives. Regardless of your opinion about on-prem versus multitenant cloud infrastructure, the adoption of DevOps is improving how quickly new features and functions are delivered at scale for end users.

There is a lot to learn from the evolution of DevOps, across the modern Internet as well as within the modern enterprise—most notably for those who work with data every day.

At its core, DevOps is about the combination of software engineering, quality assurance, and technology operations (Figure 5-1). DevOps emerged because traditional systems management (as opposed to software development management) was not adequate to meet the needs of modern, web-based application development and deployment.

gtdr 0501
Figure 5-1. DevOps in the enterprise

From DevOps to DataOps

It’s time for data engineers and data scientists to embrace a new, similar discipline—let’s ...

Get Getting Data Right now with the O’Reilly learning platform.

O’Reilly members experience live online training, plus books, videos, and digital content from nearly 200 publishers.