Data Mesh

Chapter 4. Principle of the Self-Serve Data Platform

Simplicity is about subtracting the obvious and adding the meaningful.

John Maeda

So far I have offered two fundamental shifts toward data mesh: a distributed data architecture and ownership model oriented around business domains, and data shared as a usable and valuable product. Over time, these two seemingly simple and rather intuitive shifts can have undesired consequences: duplication of efforts in each domain, increased cost of operation, and likely large-scale inconsistencies and incompatibilities across domains.

Expecting domain engineering teams to own and share analytical data as a product, in addition to building applications and maintaining digital products, raises legitimate concerns for both the practitioners and their leaders. The concerns that I often hear from leaders, at this point in the conversation, include: “How am I going to manage the cost of operating the domain data products, if every domain needs to build and own its own data?” “How do I hire the data engineers, who are already hard to find, to staff in every domain?” “This seems like a lot of overengineering and duplicate effort in each team.” “What technology do I buy to provide all the data product usability characteristics?” “How do I enforce governance in a distributed fashion to avoid chaos?” “What about copied data—how do I manage that?” And so on. Similarly, domain engineering teams and practitioners voice concerns such as, “How can we extend ...