Data Science on the Google Cloud Platform, 2nd Edition

Chapter 8. Machine Learning with BigQuery ML

BigQuery is a serverless, highly scalable data warehouse. Amazingly enough, it is also an excellent machine learning platform. This combination is very convenient since you can do machine learning without having to extract data out of the data warehouse. If your organization has a lot of privacy-sensitive or confidential data, not having extracts of data floating around in people’s projects is important for security. The auditability that BigQuery provides out of the box means that you know exactly who created the model and which data was used in which model.

Given the scalability, power, ease-of-use, and security of BigQuery ML, I recommend using it, rather than Spark, for the first machine learning model you should build when working with tabular data. In fact, as you will see in this chapter, you can get best-in-class accuracy, explainability, and prediction capabilities using BigQuery ML. Because of its connections to Vertex AI, it can be your production machine learning framework also.

All of the code snippets in this chapter are available in the folder 08_bqml of the book’s GitHub repository. See the README.md file in that directory for instructions on how to do the steps described in this chapter.

Logistic Regression

Let’s start where we left off in Chapter 7—recall that we trained a logistic regression model, added the airport code, and overwhelmed Spark in the process. So, let’s start by replicating the last working model ...