Skip to Main Content
Data Engineering with Google Cloud Platform - Second Edition
book

Data Engineering with Google Cloud Platform - Second Edition

by Adi Wijaya
April 2024
Beginner to intermediate content levelBeginner to intermediate
476 pages
12h 22m
English
Packt Publishing
Content preview from Data Engineering with Google Cloud Platform - Second Edition

4

Building Workflows for Batch Data Loading Using Cloud Composer

In data engineering, the definition of a workflow is a set of configurations to automate tasks, jobs, and their dependencies. If we are talking about database workflow, we talk about how to automate the table creation process.

The main objects in any database system are tables, and one of the main differences between an application database and a data warehouse is the creation of tables. Compared to tables in application databases, where tables are mostly static and created to support frontend applications, tables in data warehouses are dynamic. The table is the main product as it is a collection of business logic, and data flows.

In this chapter, we will learn how to create the ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Fundamentals of Data Engineering

Fundamentals of Data Engineering

Joe Reis, Matt Housley
Fundamentals of Data Engineering

Fundamentals of Data Engineering

Joe Reis, Matt Housley
Visualizing Google Cloud

Visualizing Google Cloud

Priyanka Vergadia

Publisher Resources

ISBN: 9781835080115Supplemental Content