Bioinformatics with Python Cookbook - Second Edition

There's more…

The preceding recipe was made to run on a simple configuration of Airflow, with a default SQLite database, which implies sequential execution (only one task at a time). The explicit dependency of the FS sensor on the download is only there to avoid a race condition with the basic executor. Furthermore, with other executors (that is, supported by another backend database), the sub-samplings could occur in parallel.