Implementing Data Mesh

Chapter 4. Defining the Data Mesh Architecture

This chapter will discuss the core architecture components within a Data Mesh. The chapter is organized into two major sections. First, we discuss the data-product architecture, including the components required to support a wide-ranging set of artifacts and the components required for developing, running, and operating data products. Second, we focus on the broader Data Mesh architecture that binds all data products together into a unified whole.

Data Product Architecture

Each data product in a Data Mesh is designed to be discoverable, observable, and operable, ensuring that data can be efficiently shared and utilized across different parts of an organization. Figure 4-1 illustrates the data product architecture that we will elaborate on.

There are several key groups of capabilities within the data-product architecture: an architecture for artifacts as well as architectures for development, runtime, and operations capabilities.

Figure 4-1. Data product architecture

Data Product Artifacts

The contents of a data product—the artifacts—are what make the data product valuable. Artifacts include not just data but also other objects that a DPO wants to make available for consumption.

Artifacts, as shown in Figure 4-2, range from basic datasets to complex programs and models. They add significant value to the data product, making it ...