4
ETL Data Load – A Batch-Based Solution to Ingesting Data in a Data Warehouse
In the previous chapters, we discussed various foundational concepts surrounding data engineering, starting with the different types of data engineering problems. Then, we discussed various data types, data formats, data storage, and databases. We also discussed the various platforms that are available to deploy and run data engineering solutions in production.
In this chapter, we will learn how to architect and design a batch-based solution for low to medium-volume data ingestion from a data source to a data warehouse. Here, we will be taking a real-time use case to discuss, model, and design a data warehouse for such a scenario. We will also learn how to develop ...
Get Scalable Data Architecture with Java now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
