Mapping Data Flows in Azure Data Factory: Building Scalable ETL Projects in the Microsoft Cloud

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022

M. KromerMapping Data Flows in Azure Data Factoryhttps://doi.org/10.1007/978-1-4842-8612-8_9

9. Basics of CI/CD and Pipeline Scheduling

Mark Kromer¹  

(1)

SNOHOMISH, WA, USA

 

Throughout the previous parts of this book, I sort of skipped over setting up a Git environment, implementing CI/CD processes , and scheduling of your pipelines in order to focus on the data flow logic. So let’s now spend a chapter here to dive into setting up Git and scheduling your ETL jobs. These are both topics that are critically important to establishing your ADF ecosystem and to operationalize your data flows. Source code control , CI/CD, scheduling, and managing your factory pipelines ...