book

Hands-On MLOps on Azure

Name: Hands-On MLOps on Azure
Author: Banibrata De
ISBN: 9781836200338

by Banibrata De

August 2025

Intermediate to advanced

276 pages

6h 16m

English

Packt Publishing

Read now

Unlock full access

Preface
Who this book is forWhat this book coversTo get the most out of this bookGet in touchStay Sharp in Cloud and DevOps – Join 44,000+ Subscribers of CloudPro
Part 1: Foundations of MLOps
Understanding DevOps to MLOps
From DevOps to MLOps: Bridging the operational gapDevOps: A foundation for MLOpsRevolutionizing software developmentThe DevOps–MLOps connectionKey DevOps concepts in MLOpsCI/CD for the ML lifecycleThe importance of MLOps in the AI eraPrinciples and practices of MLOpsData management in MLOpsExperiment trackingModel deployment challengesSecurity and compliance in MLOpsModel performance and maintenanceMLOps tools and technologiesBuilding an MLOps teamFaster experimentation and development of modelsDeployment of models into productionQuality assurance and end-to-end lineage trackingMLOps toolkits: Streamlining the ML lifecycle with ML CLIsTypes of ML CLIsChoosing the right ML CLICommon management tasks with ML CLIsExploring ML CLIs for different cloud providersAzure ML CLI v2AWS CLI with SageMakerGCP gcloud CLIBenefits of organized structureSummary
Training and Experimentation
Key stages in building an ML modelAML workspaceKey features of an AML workspaceKey components of a workspaceManaging workspace resourcesAML CLISetting up a virtual environmentBasic structure and usage of the AML CLIWorkspace: A closer lookJobs and experiments in AMLJobsExperimentsJobs and experiments: Why they matterData preparationSteps in data preparationWhat are the benefits of proper data preparation?Registering data in the AML workspaceHow can data be registered?Setting up an experimentCreating a simple experiment by running a jobChoosing the model/algorithmDefining the evaluation criteriaCollecting metrics and artifactsComparing modelsSelecting the best modelTracking and comparing model experiments in MLTools for trackingSetting up MLflow tracking with AzureML CLI v2Comparing jobs in an experimentRegister the best model based on metricsOptimizing modelsHyperparameter tuningTuning techniquesSweep jobsExample using the CLIEvaluation and iterationSummaryTools documentation
Part 2: Implementing MLOps
Reproducible and Reusable ML
Defining repeatable and reusable steps for data preparation, training, and scoringLearning about components and pipelines in AMLComponentsPipelinesUnderstanding ML environmentsTracking and reproducing software dependencies in projectsHands-on example – Building an ML pipeline with AML CLI, Git, and GitHub ActionsSummaryJoin the CloudPro Newsletter with 44000+ Subscribers
Model Management (Registration and Packaging)
Model metadataMetadata management using Azure Machine Learning (AML)Model registrationAML registryModel formatStandardizing the model format (MLflow)Custom model formatsChallenges and considerationsChoosing the right formatDatastoresRegistering models in actionExamples of model registration with the AML CLIModel packagingCommands for model packagingProperties of a package operationCreating a packageSummary
Model Deployment: Batch Scoring and Real-Time Web Services
Model deployment optionsReal-time inferenceImplementation in AMLDeployment infrastructureBatch inference/scoringImplementation in AMLDeployment infrastructureOnline inferencingPreparing the modelRegistering the modelScoring scriptConfiguring the environmentDeploymentInference on deploymentBatch inferencingScoring scriptConfiguring the environment for online deploymentDeployment configurationConfiguring the environment for batch deploymentAdditional concepts related to batch deploymentSummary
Capturing and Securing Governance Data for MLOps
Key governance focus areasEnsuring model integrityCompliance requirements in MLLineageTools and techniques for lineage tracking in AMLBest practices for logging and documenting lineageImplementing governance across the AML lifecycleSecuring data and lineage informationGovernance strategies for compliance and quality assuranceOperationalizing governance in MLEthical considerationsBias detection and mitigationBias detectionBias mitigationComprehensive governance in actionPutting the practice togetherSummary
Monitoring the ML Model
The purpose of monitoringMonitoring: Model performance versus infrastructureInfrastructure usage monitoringLearning about DataCollectorSetting up data collectionSetting up monitoring with collected dataKey monitoring signals in AMLInfrastructure metric monitoringEndpoint metricsDeployment metricsSummaryJoin the CloudPro Newsletter with 44000+ Subscribers

Notification and Alerting in MLOps
Understanding alerts and notifications in the MLOps contextExploring AML platform logsCreating an alertExtending alerts to multiple workspacesIntroduction to Log Analytics workspacesConfiguring centralized collectionAdvanced alertingIntegrating alerts with incident managementBest practices for alert managementSetting appropriate alert thresholdsAvoiding alert fatigueExample: Refining model deployment failure alertsSummary
Part 3: MLOps and Beyond
Automating the ML Lifecycle with ML Pipelines and GitHub Workflows
Implementing end-to-end AML pipelinesAML pipelineExpanding beyond Azure: GitHub Actions for CI/CDReal-world scenario: Multi-cloud CI/CD for ML workflowsChallenges and best practicesCommon challenges in multi-cloud ML pipelinesBest practicesSummary
Using Models in Real-world Applications
Recapping fundamental conceptsCase study 1: Demand forecasting on AzureBusiness context and requirementsImplementation architectureData pipeline Model development pipeline CI/CD pipeline Deployment and serving Monitoring and logging Feedback loop Platform-specific solutionChallenges and solutionsRegional time-series forecasting Scalability and performance Case study 2: Handwriting assistance for children on Google Cloud PlatformBusiness context and requirementsImplementation architectureData pipeline Model development pipeline CI/CD pipelineDeployment and serving Monitoring and logging Feedback loopChallenges and solutions Variability in handwriting styles Real-time inference performanceCase study 3: Real-time precision delivery on Amazon Web ServicesBusiness context and requirementsImplementation architectureData pipeline Model development pipeline CI/CD pipeline Deployment and serving Monitoring and logging Feedback loop Challenges and solutionsReal-time processing at scale Complex route optimizationSummary
Exploring Next-Gen MLOps
Introducing LLMs: New concepts and key differences from MLOpsComponents of LLM solution developmentDevelopment processReadiness for deploymentChallenges and risks in LLMOpsResponsible AIAzure RAIDeploymentAlerting and monitoringBenefits of and trends in LLM developmentsEmerging trends transforming LLMOpsPractical example: Implementing LLMOps with Azure AIBackgroundSolution development Prompt engineering and model customization RAI implementation Deployment and monitoringResults and impactFuture developments SummaryStay Sharp in Cloud and DevOps – Join 44,000+ Subscribers of CloudPro
Other Books You May Enjoy
Index

Content preview from Hands-On MLOps on Azure

7 Monitoring the ML Model

In this chapter, we will go through the critical aspects of monitoring Machine Learning (ML) models and their supporting infrastructure in AML. We begin by exploring the purpose and importance of monitoring in MLOps, highlighting its role in ensuring the continued reliability, performance, and efficiency of deployed ML solutions.

This chapter provides a comprehensive exploration of monitoring strategies essential for maintaining robust ML operations in AML. We’ll examine the critical distinction between model performance monitoring and infrastructure usage monitoring, introduce you to the DataCollector tool and its central role in tracking model behavior, and guide you through setting up data collection for your deployed ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Azure OpenAI Service for Cloud Native Applications

Publisher Resources

ISBN: 9781836200338

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Hands-On MLOps on Azure

by Banibrata De

7

Monitoring the ML Model

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.