Getting DataOps Right

Chapter 6. Building a DataOps Toolkit

Liam Cleary

The DataOps cross-functional toolkit is not a single tool or platform—nor can it ever be. Rather, it is part of a framework that prioritizes responding to change over following a plan. This means that the toolkit is inherently a collection of complementary, best of breed tools with interoperability and automation at their core design.

Interoperability

Interoperability is perhaps the biggest departure in the DataOps stack from the data integration tools or platforms of the past.

Although a good ETL platform certainly appeared to afford interoperability as a core principle, with many out-of-the-box connectors supporting hundreds of data formats and protocols (from Java Database Connectivity [JDBC] to Simple Object Access Protocol [SOAP]), this, in fact, was a reaction to the complete lack of interoperability in the set of traditional data repositories and tools.

In practice, if your ETL platform of choice did not support a native connector to a particular proprietary database, enterprise resource planning (ERP), customer relationship management (CRM), or business suite, the data was simply forfeited from any integration project. Conversely, data repositories that did not support an open data exchange format forced users to work within the confines of that data repository, often subsisting with rigid, suboptimal, worst-of-breed, tack-on solutions. The dream of being able to develop a center of excellence around a single data integration ...