July 2024
Intermediate to advanced
286 pages
7h 53m
English
Content preview from Practical Lakehouse Architecture
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Chapter 8. Lakehouse in the Real World
We are approaching the end of this book and, so far, we’ve discussed various components, design considerations, tools and technologies, and best practices for implementing a lakehouse platform. In all these discussions, we’ve looked at the ideal design approaches and optimized solutions for building a lakehouse. However, as we all know, reality is much different than the ideal world. Implementing a lakehouse in the real world comes with several unforeseen challenges. To overcome these challenges, you will have to make some practical decisions and technical adjustments based on your judgment and experience.
In this chapter, I’ll discuss what it means to build a lakehouse in the real world—a world where projects get delayed by months (if not years), requirements keep changing, budgets keep reducing, and old platforms never get decommissioned! We will discuss several technical and non-technical challenges across the planning, design, implementation, and support phases of a lakehouse project.
Building a data platform is a long journey that can take several months to years. In the sections that follow, I’ll guide you through estimating, planning, and executing your lakehouse project and help you overcome some common, practical challenges. I’ll also provide you with a list of project deliverables and reference architectures that can help you kickstart your lakehouse journey.
Delivering a Real-World Lakehouse
As discussed in the previous chapters, ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.