Data Pipelines with Apache Airflow

6 Triggering workflows

This chapter covers

• Waiting for certain conditions to be met with sensors
• Deciding how to set dependencies between tasks in different DAGs
• Executing workflows via the CLI and REST API

In chapter 3, we explored how to schedule workflows in Airflow based on a time interval. The time intervals can be given as convenience strings (e.g., "@daily"), timedelta objects (e.g., timedelta(days=3)), or cron strings (e.g., "30 14 * * *"). These are all notations to instruct the workflow to trigger at a certain time or interval. Airflow will compute the next time to run the workflow given the interval and start the first task(s) in the workflow at the next date and time.

In this chapter, we explore other ways to trigger workflows. ...