Learning Google BigQuery

Importing message data into BigQuery

To import messages into BigQuery in batch mode, you can use the gcloud utility to log all the messages retrieved from the Cloud Pub/Sub to a file. Then you can upload the file to Google Cloud Storage and import data from the file to the BigQuery table. The following section provides a walkthrough of this scenario. This section uses the same Pub/Sub topic used in the previous section and uses the same Pub/Sub subscriber for pulling messages. Publish some sample messages to the Pub/Sub topic before running the commands given in this section.

Schedule a job to call the following command at specified intervals to retrieve the message from Cloud Pub/Sub, and append it to a local file. Replace the subscriber, ...