97 Things Every Data Engineer Should Know

Chapter 38. How to Build Your Data Platform like a Product

Barr Moses and Atul Gupte

At its core, a data platform is a central repository for all data, handling the collection, cleansing, transformation, and application of data to generate business insights. For most organizations, building a data platform is no longer a nice-to-have option but a necessity, with many businesses distinguishing themselves from the competition based on their ability to glean actionable insights from their data.

Much in the same way that many view data itself as a product, data-first companies like Uber, LinkedIn, and Facebook increasingly view data platforms as products too, with dedicated engineering, product, and operational teams. Despite their ubiquity and popularity, however, data platforms are often spun up with little foresight into who is using them, how they’re being used, and what engineers and product managers can do to optimize these experiences.

Whether you’re just getting started or are in the process of scaling one, we share three best practices for avoiding these common pitfalls and building the data platform of your dreams.

Align Your Product’s Goals with the Goals of the Business

When you’re building or scaling your data platform, the first question you should ask is, how does data ...