Building an Event-Driven Data Mesh

Chapter 2. Data Mesh

A helpful elevator pitch I often heard referred to data mesh as “microservices for data”—the same principles, but you’re applying them to data instead of services. An organization using microservices will have a catalog to look up available services, including their APIs, SLAs, domain owners, security information, and access controls, along with any other organizational-specific information. The microservice catalog provides you with a view into the functions you have available to stitch together new business applications.

In both the microservice and data mesh worlds, common infrastructure services (e.g., Git, Kubernetes, containers, continuous integration, monitoring) provide self-service tooling that lets you focus on building useful business services instead of getting lost in the infrastructure and platforms. Data mesh draws a direct parallel to the microservices architecture—but with data sets, instead of services.

The benefits of a well-built data mesh include:

• Discovering trustworthy and reliable data, making it cheaper and faster to put it into use.

• Making it easier to publish new data sources, such that others can make use of them quickly and easily.

• Treating data as a first-class product, just like any other mission-critical product, including dedicated resourcing, well-defined responsibilities, SLAs, and product release cycles.

• Reducing and eventually eliminating unreliable, fragile, and expensive data pipelines and ETLs.

• Eliminating ...