Understanding Azure Data Factory: Operationalizing Big Data and Advanced Analytics Solutions

© Sudhir Rawat and Abhishek Narain 2019

Sudhir Rawat and Abhishek NarainUnderstanding Azure Data Factoryhttps://doi.org/10.1007/978-1-4842-4122-6_3

3. Data Movement

Sudhir Rawat¹  and Abhishek Narain²

(1)

Bangalore, India

(2)

Shanghai, China

 

Any extract-transform-load (ETL) or extract-load-transform (ELT) project starts with data ingestion (Figure 3-1). You should be able to connect to various sources, either in a public network or behind firewalls in a private network, and then be able to pull them onto a staging location or a destination on the cloud. In the ELT pattern for Big Data processing, you would generally dump all your data in a staging blob or data lake on the cloud, and based on the need, you would run analytical jobs/transform activities ...