14 Project: Finding the fastest way to get around NYC
This chapter covers
- Setting up an Airflow pipeline from scratch
- Structuring intermediate output data
- Developing idempotent tasks
- Implementing one operator to handle multiple similar transformations
Transportation in New York City (NYC) can be hectic. It’s always rush hour, but luckily there are more alternative ways of transportation than ever. In May 2013, Citi Bike started operating in New York City with 6,000 bikes. Over the years, Citi Bike has grown and expanded and has become a popular method of transportation in the city.
Another iconic method of transportation is the Yellow Cab taxi. Taxis were introduced in NYC in the late 1890s and have always been popular. However, in recent ...
Get Data Pipelines with Apache Airflow now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
