Building Google Cloud Platform Solutions

Exporting to BigQuery and Cloud Pub/Sub

As we saw in Chapter 10, Google Cloud Storage, there is quite a lot that we can do with logs once they land in our new bucket. However, a common use case for exported logs is to perform some immediate analysis on them, or take some action based on the log's contents. For these cases, the hourly batch process of Cloud Storage sinks is likely too slow, and it introduces an unnecessary hop to the final destination.

As an alternative, developers can choose to leverage Cloud Pub/Sub and BigQuery sinks. Logs written to both of these sink services are exported immediately. This can be critical in systems that must take immediate action on specific events, or which require the most up-to-date view of system ...