Skip to Main Content
Data Engineering with Google Cloud Platform - Second Edition
book

Data Engineering with Google Cloud Platform - Second Edition

by Adi Wijaya
April 2024
Beginner to intermediate content levelBeginner to intermediate
476 pages
12h 22m
English
Packt Publishing
Content preview from Data Engineering with Google Cloud Platform - Second Edition

6

Processing Streaming Data with Pub/Sub and Dataflow

Processing streaming data is becoming increasingly popular since this enables businesses to get real-time metrics on business operations. In this chapter, we will understand which paradigm should be used – and when – for streaming data. We will also learn how to apply transformations to streaming data using Cloud Dataflow, as well as how to store processed records in BigQuery for analysis.

Learning about streaming data is easier when we do it, so we will complete some exercises where we will create a streaming data pipeline on Google Cloud Platform (GCP). We will use two GCP services, Pub/Sub and Dataflow. Both services are essential in creating a streaming data pipeline. At the end of this ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Fundamentals of Data Engineering

Fundamentals of Data Engineering

Joe Reis, Matt Housley
Fundamentals of Data Engineering

Fundamentals of Data Engineering

Joe Reis, Matt Housley
Visualizing Google Cloud

Visualizing Google Cloud

Priyanka Vergadia

Publisher Resources

ISBN: 9781835080115Supplemental Content