Official Google Cloud Certified Professional Machine Learning Engineer Study Guide

Chapter 12Model Monitoring, Tracking, and Auditing Metadata

GOOGLE CLOUD PROFESSIONAL MACHINE LEARNING ENGINEER EXAM OBJECTIVES COVERED IN THIS CHAPTER:

• 2.3 Tracking and running ML experiments. Considerations include:
  • Choosing the appropriate Google Cloud environment for development and experimentation (e.g., Vertex AI Experiments, Kubeflow Pipelines, Vertex AI TensorBoard with TensorFlow and PyTorch) given the framework
• 5.3 Tracking and auditing metadata. Considerations include:
  • Tracking and comparing model artifacts and versions (e.g., Vertex AI Experiments, Vertex ML Metadata)
  • Hooking into model and dataset versioning
  • Model and data lineage
• 6.2 Monitoring, testing, and troubleshooting ML solutions. Considerations include:
  • Establishing continuous evaluation metrics (e.g., Vertex AI Model Monitoring, Explainable AI)
  • Monitoring for training‐serving skew
  • Monitoring for feature attribution drift
  • Monitoring model performance against baselines, simpler models, and across the time dimension

In the previous chapters you saw how to collect the data, train the model, find the best hyperparameter, and deploy the model. In this chapter, you will see how to monitor the model ...