Data Engineering with AWS - Second Edition

1

An Introduction to Data Engineering

Data engineering continues to be a fast-growing career path and a role in high demand, as data becomes ever more critical to organizations of all sizes. For those that enjoy the challenge of putting together the “puzzle pieces” that build out complex data pipelines to ingest raw data, and then transform and optimize that data for varied data consumers, it can be a really rewarding career.

In this chapter, we look at the many ways that data has become an important, and increasingly valuable, corporate asset. We also review some of the challenges that organizations face as they deal with increasing volumes of data, and how data engineers can use cloud-based services to help overcome these challenges. We then ...