Building ETL Pipelines with Python

15

Use Cases and Further Reading

As we enter the last chapter of this book, we can look back on our path of understanding, designing, and building ETL pipelines in Python. We started by creating simple ETL pipelines in standard Python, then introduced ETL-specific Python modules to enhance our pipelines and create more efficient, reliable implementations. We introduced various external tools, including Apache Airflow and the AWS ecosystem, to expand your ETL foundation into the cloud. Finally, we brought ETL design full circle by discussing best practices regarding monitoring and logging your code, as well as common pitfalls to watch out for. We can confidently say that you now possess the foundational knowledge to deal with the diverse challenges ...