book

Introducing MLOps

by Mark Treveil, Nicolas Omont, Clément Stenac, Kenji Lefevre, Du Phan, Joachim Zentici, Adrien Lavoillotte, Makoto Miyazaki, Lynn Heidmann

November 2020

Beginner to intermediate

183 pages

5h 9m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Preface
Who This Book Is ForHow This Book Is OrganizedConventions Used in This BookO’Reilly Online LearningHow to Contact UsAcknowledgments
I. MLOps: What and Why
1. Why Now and Challenges
Defining MLOps and Its ChallengesMLOps to Mitigate RiskRisk AssessmentRisk MitigationMLOps for Responsible AIMLOps for ScaleClosing Thoughts
2. People of MLOps
Subject Matter ExpertsData ScientistsData EngineersSoftware EngineersDevOpsModel Risk Manager/AuditorMachine Learning ArchitectClosing Thoughts
3. Key MLOps Features
A Primer on Machine LearningModel DevelopmentEstablishing Business ObjectivesData Sources and Exploratory Data AnalysisFeature Engineering and SelectionTraining and EvaluationReproducibilityResponsible AIProductionalization and DeploymentModel Deployment Types and ContentsModel Deployment RequirementsMonitoringDevOps ConcernsData Scientist ConcernsBusiness ConcernsIteration and Life CycleIterationThe Feedback LoopGovernanceData GovernanceProcess GovernanceClosing Thoughts
II. MLOps: How
4. Developing Models
What Is a Machine Learning Model?In TheoryIn PracticeRequired ComponentsDifferent ML Algorithms, Different MLOps ChallengesData ExplorationFeature Engineering and SelectionFeature Engineering TechniquesHow Feature Selection Impacts MLOps StrategyExperimentationEvaluating and Comparing ModelsChoosing Evaluation MetricsCross-Checking Model BehaviorImpact of Responsible AI on ModelingVersion Management and ReproducibilityClosing Thoughts
5. Preparing for Production
Runtime EnvironmentsAdaptation from Development to Production EnvironmentsData Access Before Validation and Launch to ProductionFinal Thoughts on Runtime EnvironmentsModel Risk EvaluationThe Purpose of Model ValidationThe Origins of ML Model RiskQuality Assurance for Machine LearningKey Testing ConsiderationsReproducibility and AuditabilityMachine Learning SecurityAdversarial AttacksOther VulnerabilitiesModel Risk MitigationChanging EnvironmentsInteractions Between ModelsModel MisbehaviorClosing Thoughts
6. Deploying to Production
CI/CD PipelinesBuilding ML ArtifactsWhat’s in an ML Artifact?The Testing PipelineDeployment StrategiesCategories of Model DeploymentConsiderations When Sending Models to ProductionMaintenance in ProductionContainerizationScaling DeploymentsRequirements and ChallengesClosing Thoughts
7. Monitoring and Feedback Loop
How Often Should Models Be Retrained?Understanding Model DegradationGround Truth EvaluationInput Drift DetectionDrift Detection in PracticeExample Causes of Data DriftInput Drift Detection TechniquesThe Feedback LoopLoggingModel EvaluationOnline EvaluationClosing Thoughts

8. Model Governance
Who Decides What Governance the Organization Needs?Matching Governance with Risk LevelCurrent Regulations Driving MLOps GovernancePharmaceutical Regulation in the US: GxPFinancial Model Risk Management RegulationGDPR and CCPA Data Privacy RegulationsThe New Wave of AI-Specific RegulationsThe Emergence of Responsible AIKey Elements of Responsible AIElement 1: DataElement 2: BiasElement 3: InclusivenessElement 4: Model Management at ScaleElement 5: GovernanceA Template for MLOps GovernanceStep 1: Understand and Classify the Analytics Use CasesStep 2: Establish an Ethical PositionStep 3: Establish ResponsibilitiesStep 4: Determine Governance PoliciesStep 5: Integrate Policies into the MLOps ProcessStep 6: Select the Tools for Centralized Governance ManagementStep 7: Engage and EducateStep 8: Monitor and RefineClosing Thoughts
III. MLOps: Real-World Examples
9. MLOps in Practice: Consumer Credit Risk Management
Background: The Business Use CaseModel DevelopmentModel Bias ConsiderationsPrepare for ProductionDeploy to ProductionClosing Thoughts
10. MLOps in Practice: Marketing Recommendation Engines
The Rise of Recommendation EnginesThe Role of Machine LearningPush or Pull?Data PreparationDesign and Manage ExperimentsModel Training and DeploymentScalability and CustomizabilityMonitoring and Retraining StrategyReal-Time ScoringAbility to Turn Recommendations On and OffPipeline Structure and Deployment StrategyMonitoring and FeedbackRetraining ModelsUpdating ModelsRuns Overnight, Sleeps During DaytimeOption to Manually Control ModelsOption to Automatically Control ModelsMonitoring PerformanceClosing Thoughts
11. MLOps in Practice: Consumption Forecast
Power SystemsData CollectionProblem Definition: Machine Learning, or Not Machine Learning?Spatial and Temporal ResolutionImplementationModelingDeploymentMonitoringClosing Thoughts
Index

Content preview from Introducing MLOps

Part I. MLOps: What and Why

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781492083283Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Introducing MLOps

by Mark Treveil, Nicolas Omont, Clément Stenac, Kenji Lefevre, Du Phan, Joachim Zentici, Adrien Lavoillotte, Makoto Miyazaki, Lynn Heidmann

Part I. MLOps: What and Why

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.