book

Engineering MLOps

Name: Engineering MLOps
Author: Emmanuel Raj
ISBN: 9781800562882

by Emmanuel Raj

April 2021

Beginner to intermediate

370 pages

7h 59m

English

Packt Publishing

Read now

Unlock full access

Engineering MLOps
ContributorsAbout the authorAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchReviews
Section 1: Framework for Building Machine Learning Models
Chapter 1: Fundamentals of an MLOps Workflow
The evolution of infrastructure and software developmentThe rise of machine learning and deep learningThe end of Moore's lawAI-centric applicationsSoftware development evolutionTraditional software development challengesTrends of ML adoption in software development Understanding MLOps Concepts and workflow of MLOps Discussing a use case Summary
Chapter 2: Characterizing Your Machine Learning Problem
The ML solution development processTypes of ML modelsLearning modelsHybrid modelsStatistical models HITL modelsStructuring your MLOpsSmall data opsBig data opsHybrid MLOpsLarge-scale MLOpsAn implementation roadmap for your solution Phase 1 – ML developmentPhase 2 – Transition to operationsPhase 3 – OperationsProcuring data, requirements, and toolsDataRequirementsTools and infrastructureDiscussing a real-life business problemSummary
Chapter 3: Code Meets Data
Business problem analysis and categorizing the problemSetting up the resources and toolsInstalling MLflowAzure Machine LearningAzure DevOpsJupyterHub10 principles of source code management for MLWhat is good data for ML?Data preprocessingData quality assessmentCalibrating missing dataLabel encoding New feature – Future_weather_conditionData correlations and filteringTime series analysisData registration and versioningToward the ML PipelineFeature Store Summary
Chapter 4: Machine Learning Pipelines
Going through the basics of ML pipelinesData ingestion and feature engineering Data ingestion (training dataset)Machine learning training and hyperparameter optimizationSupport Vector MachineRandom Forest classifierModel testing and defining metricsTesting the SVM classifierTesting the Random Forest classifierModel packaging Registering models and production artifacts Registering production artifactsSummary
Chapter 5: Model Evaluation and Packaging
Model evaluation and interpretability metricsLearning models' metricsHybrid models' metricsStatistical models' metricsHITL model metricsProduction testing methodsBatch testingA/B testingStage test or shadow testTesting in CI/CDWhy package ML models?Portability InferenceInteroperability Deployment agnosticity How to package ML modelsSerialized filesPacketizing or containerizing Microservice generation and deploymentInference ready models Connecting to the workspace and importing model artifactsLoading model artifacts for inferenceSummary
Section 2: Deploying Machine Learning Models at Scale
Chapter 6: Key Principles for Deploying Your ML System
ML in research versus productionDataFairnessInterpretabilityPerformancePriority Understanding the types of ML inference in productionDeployment targetsMapping the infrastructure for our solutionHands-on deployment (for the business problem)Deploying the model on ACIDeploying the model on Azure Kubernetes Service (AKS)Deploying the service using MLflow Understanding the need for continuous integration and continuous deploymentSummary

Chapter 7: Building Robust CI/CD Pipelines
Continuous integration, delivery, and deployment in MLOpsContinuous integrationContinuous deliveryContinuous deploymentSetting up a CI/CD pipeline and the test environment (using Azure DevOps)Creating a service principalInstalling the extension to connect to the Azure ML workspaceSetting up a continuous integration and deployment pipeline for the test environmentConnecting artifacts to the pipelineSetting up a test environmentPipeline execution and testing Pipeline execution triggersSummary
Chapter 8: APIs and Microservice Management
Introduction to APIs and microservicesWhat is an Application Programming Interface (API)?MicroservicesThe need for microservices for MLHypothetical use caseStage 1 – Proof of concept (a monolith)Stage 2 – Production (microservices)Old is gold – REST API-based microservicesHands-on implementation of serving an ML model as an APIAPI design and development Developing a microservice using DockerTesting the API Summary
Chapter 9: Testing and Securing Your ML Solution
Understanding the need for testing and securing your ML applicationTesting your ML solution by designData testingModel testingPre-training testsPost-training testsHands-on deployment and inference testing (a business use case)Securing your ML solution by designTypes of attacksSummary
Chapter 10: Essentials of Production Release
Setting up the production infrastructureAzure Machine Learning workspaceAzure Machine Learning SDKSetting up our production environment in the CI/CD pipelineTesting our production-ready pipelineConfiguring pipeline triggers for automationSetting up a Git trigger Setting up an Artifactory triggerSetting up a Schedule triggerPipeline release managementToward continuous monitoringSummary
Section 3: Monitoring Machine Learning Models in Production
Chapter 11: Key Principles for Monitoring Your ML System
Understanding the key principles of monitoring an ML systemModel driftModel bias Model transparency Model compliance Explainable AI Monitoring in the MLOps workflowUnderstanding the Explainable Monitoring FrameworkMonitor Analyze GovernEnabling continuous monitoring for the serviceSummary
Chapter 12: Model Serving and Monitoring
Serving, monitoring, and maintaining models in productionExploring different modes of serving ML models Serving the model as a batch serviceServing the model to a human userServing the model to a machine Implementing the Explainable Monitoring framework Monitoring your ML systemAnalyzing your ML systemGoverning your ML system Summary
Chapter 13: Governing the ML System for Continual Learning
Understanding the need for continual learning Continual learning The need for continual learning Explainable monitoring – governance Alerts and actionsModel QA and controlModel auditing and reports Enabling model retraining Manual model retraining Automated model retraining Maintaining the CI/CD pipeline Summary
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youLeave a review - let other readers know what you think

Content preview from Engineering MLOps

Chapter 11: Key Principles for Monitoring Your ML System

In this chapter, we will learn about the fundamental principles that are essential for monitoring your machine learning (ML) models in production. You will learn how to build trustworthy and Explainable AI solutions using the Explainable Monitoring Framework. The Explainable Monitoring Framework can be used to build functional monitoring pipelines so that you can monitor ML models in production, analyze application and model performance, and govern ML systems. The goal of monitoring ML systems is to enable trust, transparency, and explainability in order to increase business impact. We will learn about this by looking at some real-world examples.

Understanding the principles mentioned ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781800562882

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Engineering MLOps

by Emmanuel Raj

Chapter 11: Key Principles for Monitoring Your ML System

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.