Implementing Data Mesh

Part II. Designing, Building, and Deploying Data Mesh

Now that you understand the basic concepts, you can start your journey of designing and building Data Mesh components. Most of the work might be done by software engineers under the supervision of data engineers and architects. This section will target both groups of engineers (software and data), so terms that may not be familiar to data engineers will be explained.

Chapter 4, “Defining the Data Mesh Architecture”, explores the core components of a Data Mesh, focusing on the architecture of data products as well as the broader Data Mesh architecture and highlighting how various artifacts and development, runtime, and operational capabilities come together to create discoverable, observable, and operable data products. The chapter also delves into how these components are integrated through Data Mesh backbone services, marketplaces, and registries, using Climate Quantum Inc. as a case study to illustrate the practical application of these principles for managing complex climate data.

Chapter 5, “Driving Data Products with Data Contracts”, discusses the implementation of data products, emphasizing the role of data contracts in establishing trust by ensuring data quality and service levels and using examples from Climate Quantum Inc. to illustrate these concepts. The chapter explores the principles of product thinking, details the elements of data contracts, and introduces the data quality of service (data QoS) framework for ...