© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2023
R. SoniSnowflake SnowPro™ Advanced Architect Certification CompanionCertification Study Companion Serieshttps://doi.org/10.1007/978-1-4842-9262-4_12

12. Continuous Data Pipelines

Ruchi Soni1  
(1)
New Delhi, India
 

For organizations to do effective analysis, it is important to move data from different sources into a single data store. There are multiple stages involved to support this data movement, including capturing data in its raw format in the first stage followed by applying various transformations to the data to be consumed by users.

In general, there are three ways to load data: batches, micro-batches, and near real time. Batch processing refers to loading ...

Get Snowflake SnowPro™ Advanced Architect Certification Companion: Hands-on Preparation and Practice now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.