16 Airflow on AWS
This chapter covers
- Designing a deployment strategy for AWS using ECS, S3, EFS and RDS services
- An overview of several AWS-specific hooks and operators
- Demonstrating how to use AWS-specific hooks and operators with a use case
After our brief introduction in the previous chapter, this chapter will dive further into how to deploy and integrate Airflow with cloud services in Amazon AWS. First, we’ll start by designing an Airflow deployment by mapping the different components of Airflow to AWS services. Then we’ll explore some of the hooks and operators that Airflow provides for integrating with several key AWS services. Finally, we’ll show how to use these AWS-specific operators and hooks to implement a use case for generating ...
Get Data Pipelines with Apache Airflow now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
