4 Orchestration
This chapter covers
- Building a data ingestion pipeline
- Introducing Azure Data Factory
- DevOps for Azure Data Factory
- Monitoring with Azure Monitor
In this chapter, we’ll look at the final pieces of core infrastructure for our data platform: orchestration and monitoring. DevOps is where we store all our code and configurations and from which we deploy our services. The storage layer is where we ingest data and on top of which we run our workloads. The orchestration layer handles data movement and all other automated processing. Figure 4.1 highlights the platform layer we’ll focus on in this chapter.
Figure 4.1 The orchestration ...
Get Data Engineering on Azure now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
