Building Data Integration Solutions

Chapter 1. Introduction to Data Integration

This chapter provides an overview of what data integration actually is, what role it plays in the overall data life cycle, and how it relates to an organization’s data strategy. It aims to equip you with the basic understanding necessary to effectively implement a data integration solution and align the solution to broader organizational objectives.

Data Integration and Data Management

Data life cycle management encompasses all the disciplines related to obtaining and maintaining value from data. Effective data management ensures that data is accurate, available, and accessible and is a primary component in the decision-making process. You may have heard the term DataOps, which is a style of data management that focuses on collaboration between stakeholders throughout the data life cycle, much the same way that DevOps is centered around collaboration between software development teams. DataOps emphasizes automation, quality, and continuous delivery in data processes, similar to DevOps in software development.

I prefer to partition the management of the data life cycle into three segments. As you can see in Figure 1-1, the segments include data integration, data analytics, and data governance. Each has distinct objectives and consists of lower-level processes that combine to form data pipelines.¹ The lower-level processes sometime live in the gray area between the components.

Data analytics and data governance are no less important than ...