Azure Data Factory by Example: Practical Implementation for Data Engineers

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2024

R. SwinbankAzure Data Factory by Examplehttps://doi.org/10.1007/979-8-8688-0218-8_7

7. Data Flows

Richard Swinbank¹  

(1)

Tewkesbury, UK

 

The Copy activity is a powerful tool for moving data between storage systems, but it has limited support for data transformation. Columns can be added to the activity’s source configuration, or removed by excluding them from the source to sink mapping, but the activity does not support manipulation of individual rows or allow data sources to be combined or separated.

The Azure platform offers a number of tools based on Apache Spark, including Azure Databricks, Azure Synapse Analytics Spark pools, and Azure HDInsight. Apache ...