Chapter 5: Data Collection Stage – The Bronze Layer
In the previous chapters, we discussed many theories involving data lakes, their architectures, and pipelines as a method in which to create data lakes. Now that you have a fair understanding of these topics, it is the perfect time to begin creating our actual data lakehouse.
In this chapter, we will cover the following topics:
- Architecting the Electroniz data lake
- Understanding the bronze layer
- Configuring data sources
- Configuring data destinations
- Building the ingestion pipelines
- Testing the ingestion pipelines
Architecting the Electroniz data lake
In the previous chapter, Chapter 4, Understanding Data Pipelines, we introduced the sample lakehouse project for a leading big-box store named ...
Get Data Engineering with Apache Spark, Delta Lake, and Lakehouse now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
