4
Building Workflows for Batch Data Loading Using Cloud Composer
In data engineering, the definition of a workflow is a set of configurations to automate tasks, jobs, and their dependencies. If we are talking about database workflow, we talk about how to automate the table creation process.
The main objects in any database system are tables, and one of the main differences between an application database and a data warehouse is the creation of tables. Compared to tables in application databases, where tables are mostly static and created to support frontend applications, tables in data warehouses are dynamic. The table is the main product as it is a collection of business logic, and data flows.
In this chapter, we will learn how to create the ...
Get Data Engineering with Google Cloud Platform - Second Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
