Engineering MLOps

Engineering MLOps

by Emmanuel Raj
Released April 2021
Publisher(s): Packt Publishing
ISBN: 9781800562882

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Get up and running with machine learning life cycle management and implement MLOps in your organization

Key Features

  • Become well-versed with MLOps techniques to monitor the quality of machine learning models in production
  • Explore a monitoring framework for ML models in production and learn about end-to-end traceability for deployed models
  • Perform CI/CD to automate new implementations in ML pipelines

Book Description

Engineering MLps presents comprehensive insights into MLOps coupled with real-world examples in Azure to help you to write programs, train robust and scalable ML models, and build ML pipelines to train and deploy models securely in production.

The book begins by familiarizing you with the MLOps workflow so you can start writing programs to train ML models. Then you'll then move on to explore options for serializing and packaging ML models post-training to deploy them to facilitate machine learning inference, model interoperability, and end-to-end model traceability. You'll learn how to build ML pipelines, continuous integration and continuous delivery (CI/CD) pipelines, and monitor pipelines to systematically build, deploy, monitor, and govern ML solutions for businesses and industries. Finally, you'll apply the knowledge you've gained to build real-world projects.

By the end of this ML book, you'll have a 360-degree view of MLOps and be ready to implement MLOps in your organization.

What you will learn

  • Formulate data governance strategies and pipelines for ML training and deployment
  • Get to grips with implementing ML pipelines, CI/CD pipelines, and ML monitoring pipelines
  • Design a robust and scalable microservice and API for test and production environments
  • Curate your custom CD processes for related use cases and organizations
  • Monitor ML models, including monitoring data drift, model drift, and application performance
  • Build and maintain automated ML systems

Who this book is for

This MLOps book is for data scientists, software engineers, DevOps engineers, machine learning engineers, and business and technology leaders who want to build, deploy, and maintain ML systems in production using MLOps principles and techniques. Basic knowledge of machine learning is necessary to get started with this book.

Table of contents

  1. Engineering MLOps
  2. Contributors
  3. About the author
  4. About the reviewers
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Download the color images
    6. Conventions used
    7. Get in touch
    8. Reviews
  6. Section 1: Framework for Building Machine Learning Models
  7. Chapter 1: Fundamentals of an MLOps Workflow
    1. The evolution of infrastructure and software development
      1. The rise of machine learning and deep learning
      2. The end of Moore's law
      3. AI-centric applications
      4. Software development evolution
    2. Traditional software development challenges
    3. Trends of ML adoption in software development
    4. Understanding MLOps
    5. Concepts and workflow of MLOps
      1. Discussing a use case
    6. Summary
  8. Chapter 2: Characterizing Your Machine Learning Problem
    1. The ML solution development process
    2. Types of ML models
      1. Learning models
      2. Hybrid models
      3. Statistical models
      4. HITL models
    3. Structuring your MLOps
      1. Small data ops
      2. Big data ops
      3. Hybrid MLOps
      4. Large-scale MLOps
    4. An implementation roadmap for your solution
      1. Phase 1 – ML development
      2. Phase 2 – Transition to operations
      3. Phase 3 – Operations
    5. Procuring data, requirements, and tools
      1. Data
      2. Requirements
    6. Tools and infrastructure
    7. Discussing a real-life business problem
    8. Summary
  9. Chapter 3: Code Meets Data
    1. Business problem analysis and categorizing the problem
    2. Setting up the resources and tools
      1. Installing MLflow
      2. Azure Machine Learning
      3. Azure DevOps
      4. JupyterHub
    3. 10 principles of source code management for ML
    4. What is good data for ML?
    5. Data preprocessing
      1. Data quality assessment
      2. Calibrating missing data
      3. Label encoding 
      4. New feature – Future_weather_condition
      5. Data correlations and filtering
      6. Time series analysis
    6. Data registration and versioning
    7. Toward the ML Pipeline
      1. Feature Store
    8. Summary
  10. Chapter 4: Machine Learning Pipelines
    1. Going through the basics of ML pipelines
    2. Data ingestion and feature engineering
      1. Data ingestion (training dataset)
    3. Machine learning training and hyperparameter optimization
      1. Support Vector Machine
      2. Random Forest classifier
    4. Model testing and defining metrics
      1. Testing the SVM classifier
      2. Testing the Random Forest classifier
    5. Model packaging
    6. Registering models and production artifacts
      1. Registering production artifacts
    7. Summary
  11. Chapter 5: Model Evaluation and Packaging
    1. Model evaluation and interpretability metrics
      1. Learning models' metrics
      2. Hybrid models' metrics
      3. Statistical models' metrics
      4. HITL model metrics
    2. Production testing methods
      1. Batch testing
      2. A/B testing
      3. Stage test or shadow test
      4. Testing in CI/CD
    3. Why package ML models?
      1. Portability
      2. Inference
      3. Interoperability
      4. Deployment agnosticity
    4. How to package ML models
      1. Serialized files
      2. Packetizing or containerizing
      3. Microservice generation and deployment
    5. Inference ready models
      1. Connecting to the workspace and importing model artifacts
      2. Loading model artifacts for inference
    6. Summary
  12. Section 2: Deploying Machine Learning Models at Scale
  13. Chapter 6: Key Principles for Deploying Your ML System
    1. ML in research versus production
      1. Data
      2. Fairness
      3. Interpretability
      4. Performance
      5. Priority
    2. Understanding the types of ML inference in production
      1. Deployment targets
      2. Mapping the infrastructure for our solution
    3. Hands-on deployment (for the business problem)
      1. Deploying the model on ACI
      2. Deploying the model on Azure Kubernetes Service (AKS)
      3. Deploying the service using MLflow
    4. Understanding the need for continuous integration and continuous deployment
    5. Summary
  14. Chapter 7: Building Robust CI/CD Pipelines
    1. Continuous integration, delivery, and deployment in MLOps
      1. Continuous integration
      2. Continuous delivery
      3. Continuous deployment
    2. Setting up a CI/CD pipeline and the test environment (using Azure DevOps)
      1. Creating a service principal
      2. Installing the extension to connect to the Azure ML workspace
      3. Setting up a continuous integration and deployment pipeline for the test environment
      4. Connecting artifacts to the pipeline
      5. Setting up a test environment
    3. Pipeline execution and testing
    4. Pipeline execution triggers
    5. Summary
  15. Chapter 8: APIs and Microservice Management
    1. Introduction to APIs and microservices
      1. What is an Application Programming Interface (API)?
      2. Microservices
    2. The need for microservices for ML
      1. Hypothetical use case
      2. Stage 1 – Proof of concept (a monolith)
      3. Stage 2 – Production (microservices)
    3. Old is gold – REST API-based microservices
    4. Hands-on implementation of serving an ML model as an API
      1. API design and development
    5. Developing a microservice using Docker
    6. Testing the API
    7. Summary
  16. Chapter 9: Testing and Securing Your ML Solution
    1. Understanding the need for testing and securing your ML application
    2. Testing your ML solution by design
      1. Data testing
      2. Model testing
      3. Pre-training tests
      4. Post-training tests
    3. Hands-on deployment and inference testing (a business use case)
    4. Securing your ML solution by design
      1. Types of attacks
    5. Summary
  17. Chapter 10: Essentials of Production Release
    1. Setting up the production infrastructure
      1. Azure Machine Learning workspace
      2. Azure Machine Learning SDK
    2. Setting up our production environment in the CI/CD pipeline
    3. Testing our production-ready pipeline
    4. Configuring pipeline triggers for automation
      1. Setting up a Git trigger
      2. Setting up an Artifactory trigger
      3. Setting up a Schedule trigger
    5. Pipeline release management
    6. Toward continuous monitoring
    7. Summary
  18. Section 3: Monitoring Machine Learning Models in Production
  19. Chapter 11: Key Principles for Monitoring Your ML System
    1. Understanding the key principles of monitoring an ML system
      1. Model drift
      2. Model bias
      3. Model transparency
      4. Model compliance
      5. Explainable AI
    2. Monitoring in the MLOps workflow
    3. Understanding the Explainable Monitoring Framework
      1. Monitor
      2. Analyze
      3. Govern
    4. Enabling continuous monitoring for the service
    5. Summary
  20. Chapter 12: Model Serving and Monitoring
    1. Serving, monitoring, and maintaining models in production
    2. Exploring different modes of serving ML models
      1. Serving the model as a batch service
      2. Serving the model to a human user
      3. Serving the model to a machine
    3. Implementing the Explainable Monitoring framework
      1. Monitoring your ML system
      2. Analyzing your ML system
    4. Governing your ML system
    5. Summary
  21. Chapter 13: Governing the ML System for Continual Learning
    1. Understanding the need for continual learning
      1. Continual learning
      2. The need for continual learning
    2. Explainable monitoring – governance
      1. Alerts and actions
      2. Model QA and control
      3. Model auditing and reports
    3. Enabling model retraining
      1. Manual model retraining
      2. Automated model retraining
    4. Maintaining the CI/CD pipeline
    5. Summary
    6. Why subscribe?
  22. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Leave a review - let other readers know what you think

Product information

  • Title: Engineering MLOps
  • Author(s): Emmanuel Raj
  • Release date: April 2021
  • Publisher(s): Packt Publishing
  • ISBN: 9781800562882