Skip to Main Content
Data Engineering with AWS
book

Data Engineering with AWS

by Gareth Eagar
December 2021
Beginner to intermediate content levelBeginner to intermediate
482 pages
11h 27m
English
Packt Publishing
Content preview from Data Engineering with AWS

Chapter 6: Ingesting Batch and Streaming Data

Having developed a high-level architecture of our data pipeline, we can now dive deep into the varied components of the architecture. We will start with data ingestion so that in the hands-on section of this chapter, we can ingest data that we can use for the hands-on activities in future chapters.

Data engineers are often faced with the challenge of the five Vs of data. These are the variety of data (the diverse types and formats of data); the volume of data (the size of the dataset); the velocity of the data (how quickly the data is generated and needs to be ingested); the veracity or validity of the data (the quality, completeness, and credibility of data); and finally, the value of data (the ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Data Engineering with AWS - Second Edition

Data Engineering with AWS - Second Edition

Gareth Eagar
Data Science on AWS

Data Science on AWS

Chris Fregly, Antje Barth
Data Engineering with Python and AWS Lambda LiveLessons

Data Engineering with Python and AWS Lambda LiveLessons

Noah Gift, Robert Jordan, Kennedy Behrman

Publisher Resources

ISBN: 9781800560413Supplemental Content