Chapter 2: Data Ingestion
Data ingestion is the process of moving data from disparate operational systems to a central location such as a data warehouse or a data lake to be processed and made conducive for data analytics. It is the first step of the data analytics process and is necessary for creating centrally accessible, persistent storage, where data engineers, data scientists, and data analysts can access, process, and analyze data to generate business analytics.
You will be introduced to the capabilities of Apache Spark as a data ingestion engine for both batch and real-time processing. Various data sources supported by Apache Spark and how to access them using Spark's DataFrame interface will be presented.
Additionally, you will learn ...
Get Essential PySpark for Scalable Data Analytics now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
