Streaming inserts
As an alternative to file-based loading techniques, Google BigQuery supports streaming inserts through the BigQuery API. Streaming data into BigQuery has several advantages and disadvantages when compared to file uploading. The primary advantage to streaming inserts is that data is immediately available for querying. However, unlike file uploads, streaming inserts are billable operations, costing $0.01 per inserted MB.
In order to support high-throughput streaming inserts, BigQuery buffers records before insertion. Under certain circumstances buffered records may be temporarily unavailable for querying, however BigQuery will provide warnings under such conditions in API responses. Additionally, data streamed into BigQuery ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access