Deciphering Data Architectures

Preface

I’ve been in information technology (IT) for nearly 40 years. I’ve worked at companies of all different sizes, I’ve worked as a consultant, and I’ve owned my own company. For the last 9 years, I have been at Microsoft as a data architect, and for the last 15 years, I have been involved with data warehousing. I’ve spoken about data thousands of times, to customers and groups.

During my career, I have seen many data architectures come and go. I’ve seen too many companies argue over the best approach and end up building the wrong data architecture—a mistake that can cost them millions of dollars and months of time, putting them well behind their competitors.

What’s more, data architectures are complex. I’ve seen firsthand that most people are unclear on the concepts involved, if they’re aware of them at all. Everyone seems to be throwing around terms like data mesh, data warehouse, and data lakehouse—but if you ask 10 people what a data mesh is, you will get 11 different answers.

Where do you even start? Are these just buzzwords with a lot of hype but little substance, or are they viable approaches? They may sound great in theory, but how practical are they? What are the pros and cons of each architecture?

None of the architectures discussed in this book is “wrong.” They all have a place, but only in certain use cases. No one architecture applies to every situation, so this book is not about convincing you to choose one architecture over the others. Instead, you will get ...