BigQuery for Data Warehousing: Managed Data Analysis in the Google Cloud

© Mark Mucchetti 2020

M. MucchettiBigQuery for Data Warehousinghttps://doi.org/10.1007/978-1-4842-6186-6_6

6. Streaming Data into the Warehouse

Mark Mucchetti¹ 

(1)

Santa Monica, CA, USA

 

In the last chapter, we covered myriad ways to take your data and load it into your BigQuery data warehouse. Another significant way of getting your data into BigQuery is to stream it. In this chapter, we will cover the pros and cons of streaming data, when you might want to use it, and how to do it.

Streaming bears little resemblance to loading. Most notably, there’s no way to activate it using the UI in the BigQuery console. Consequently, we’ll be using Python for our examples. In practice, you could run this code inside a Google Compute Engine (GCE) or Google Cloud ...