4
Ingesting and Transforming Data
Welcome to the next major section of the book. In Chapter 3, Designing and Implementing Data Exploration Layer, you learned about implementing the serving layer and saw how data is shared between services such as Synapse SQL and Spark.
In this chapter, you will focus on designing and developing data processing systems. This will include an examination of data transformation—that is, the process of transforming your data from its raw format to a more useful format that can be used by downstream tools and projects utilizing services such as Spark, SQL, and Azure Data Factory (ADF), reading data using different file formats and encodings, and data cleansing.
Note
This chapter primarily focuses on the Ingest and ...
Get Azure Data Engineer Associate Certification Guide - Second Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
