Video description
Quantopian integrates financial data from vendors around the globe. As the scope of its operations outgrew cron, the company turned to Apache Airflow, a distributed scheduler and task executor. James Meickle explains how in less than six months, Quantopian was able to rearchitect brittle crontabs into resilient, recoverable pipelines defined in code to which anyone could contribute.
Product information
- Title: How Quantopian rearchitected brittle crontabs into resilient pipelines with Apache Airflow
- Author(s):
- Release date: September 2019
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 0636920452669
You might also like
video
Data Pipelines with Apache Airflow, video edition
An Airflow bible. Useful for all kinds of users, from novice to expert. Rambabu Posa, Sai …
audiobook
Data Pipelines with Apache Airflow
An Airflow bible. Useful for all kinds of users, from novice to expert. Rambabu Posa, Sai …
book
Data Pipelines with Apache Airflow
A successful pipeline moves data efficiently, minimizing pauses and blockages between tasks, keeping every process along …
book
Modern Data Engineering with Apache Spark: A Hands-On Guide for Building Mission-Critical Streaming Applications
Leverage Apache Spark within a modern data engineering ecosystem. This hands-on guide will teach you how …
