book

Implementing MLOps in the Enterprise

by Yaron Haviv, Noah Gift

December 2023

Intermediate to advanced

377 pages

9h 21m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Who This Book Is ForNavigating This BookConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsYaronNoah
What Is MLOps?MLOps in the EnterpriseUnderstanding ROI in Enterprise SolutionsUnderstanding Risk and Uncertainty in the EnterpriseMLOps Versus DevOpsWhat Isn’t MLOps?Mainstream Definitions of MLOpsWhat Is ML Engineering?MLOps and Business IncentivesMLOps in the CloudKey Cloud Development EnvironmentsThe Key Players in Cloud ComputingMLOps On-PremisesMLOps in Hybrid EnvironmentsEnterprise MLOps StrategyConclusionCritical Thinking Discussion QuestionsExercises
Getting StartedChoose Your AlgorithmDesign Your PipelinesData Collection and PreparationData Storage and IngestionData Exploration and PreparationData LabelingFeature StoresModel Development and TrainingWriting and Maintaining Production ML CodeTracking and Comparing Experiment ResultsDistributed Training and Hyperparameter OptimizationBuilding and Testing Models for ProductionDeployment (and Online ML Services)From Model Endpoints to Application PipelinesOnline Data PreparationContinuous Model and Data MonitoringMonitoring Data and Concept DriftMonitoring Model Performance and AccuracyThe Strategy of Pretrained ModelsBuilding an End-to-End Hugging Face ApplicationFlow Automation (CI/CD for ML)ConclusionCritical Thinking Discussion QuestionsExercises
Identifying the Business Use Case and GoalsFinding the AI Use CaseDefining Goals and Evaluating the ROIHow to Build a Successful ML ProjectApproving and Prototyping the ProjectScaling and Productizing ProjectsProject Structure and LifecycleML Project Example from A to ZExploratory Data AnalysisData and Model Pipeline DevelopmentApplication Pipeline DevelopmentScaling and Productizing the ProjectCI/CD and Continuous OperationsConclusionCritical Thinking Discussion QuestionsExercises
Data Versioning and LineageHow It WorksCommon ML Data Versioning ToolsData Preparation and Analysis at ScaleStructured and Unstructured Data TransformationsDistributed Data Processing ArchitecturesInteractive Data ProcessingBatch Data ProcessingStream ProcessingStream Processing FrameworksFeature StoresFeature Store Architecture and UsageIngestion and Transformation ServiceFeature StorageFeature Retrieval (for Training and Serving)Feature Stores Solutions and Usage ExampleUsing Feast Feature StoreUsing MLRun Feature StoreConclusionCritical Thinking Discussion QuestionsExercises
AutoMLRunning, Tracking, and Comparing ML JobsExperiment TrackingSaving Essential Metadata with the Model ArtifactsComparing ML Jobs: An Example with MLflowHyperparameter TuningAuto-LoggingMLOps Automation: AutoMLOpsExample: Running and Tracking ML Jobs Using Azure DatabricksHandling Training at ScaleBuilding and Running Multi-Stage WorkflowsManaging Computation Resources EfficientlyConclusionCritical Thinking Discussion QuestionsExercises
Model Registry and ManagementSolution ExamplesSageMaker ExampleMLflow ExampleMLRun ExampleModel ServingAmazon SageMakerSeldon CoreMLRun ServingAdvanced Serving and Application PipelinesImplementing Scalable Application PipelinesModel Routing and EnsemblesModel Optimization and ONNXData and Model MonitoringIntegrated Model Monitoring SolutionsStandalone Model Monitoring SolutionsModel RetrainingWhen to Retrain Your ModelsStrategies for Data RetrainingModel Retraining in the MLOps PipelineDeployment StrategiesMeasuring the Business ImpactConclusionCritical Thinking Discussion QuestionsExercises
Exploratory Data AnalysisInteractive Data PreparationPreparing the Credit Transaction DatasetPreparing the User Events (Activities) DatasetExtracting Labels and Training a ModelData Ingestion and Preparation Using a Feature StoreBuilding the Credit Transactions Data Pipeline (Feature Set)Building the User Events Data Pipeline (FeatureSet)Building the Target Labels Data Pipeline (FeatureSet)Ingesting Data into the Feature StoreModel Training and Validation PipelineCreating and Evaluating a Feature VectorBuilding and Running an Automated Training and Validation PipelineReal-Time Application PipelineDefining a Custom Model Serving ClassBuilding an Application Pipeline with Enrichment and EnsembleTesting the Application Pipeline LocallyDeploying and Testing the Real-Time Application PipelineModel MonitoringCI/CD and Continuous OperationsConclusionCritical Thinking Discussion QuestionsExercises
Distributed Deep LearningHorovodRayData Gathering, Labeling, and Monitoring in DLData Labeling Pitfalls to AvoidData Labeling Best PracticesData Labeling SolutionsUsing Foundation Models as LabelersMonitoring DL Models with Unstructured DataBuild Versus Buy Deep Learning ModelsFoundation Models, Generative AI, LLMsRisks and Challenges with Generative AIMLOps Pipelines for Efficiently Using and Customizing LLMsApplication Example: Fine-Tuning an LLM ModelConclusionCritical Thinking Discussion QuestionsExercises
ML Problem Framing with Time SeriesNavigating Time Series Analysis with AWSDiving into Time Series with DeepAR+Time Series with the GCP BigQuery and SQLBuild Versus Buy for MLOps NLP ProblemsBuild Versus Buy: The Hugging Face ApproachExploring Natural Language Processing with AWSExploring NLP with OpenAIVideo Analysis, Image Classification, and Generative AIImage Classification Techniques with CreateMLComposite AIGetting Started with Serverless for Composite AIUse Cases of Composite AI with ServerlessConclusionCritical Thinking Discussion QuestionsExercises

The Case for Rust for MLOpsLeveling Up with Rust, GitHub Copilot, and CodespacesIn the Beginning Was the Command LineGetting Started with Rust for MLOpsUsing PyTorch and Hugging Face with RustUsing Rust to Build Tools for MLOpsBuilding Containerized Rust Command-Line ToolsGPU PyTorch WorkflowsUsing TensorFlow RustDoing k-means Clustering with RustFinal Notes on RustRuff Linterrust-new-project-templateConclusionCritical Thinking Discussion QuestionsExercises

Content preview from Implementing MLOps in the Enterprise

Chapter 9. Solutions for Advanced Data Types

This chapter delves into the intricacies of data analysis and interpretation, focusing on modern techniques and approaches in time series analysis, NLP, video, and image classification. It aims to comprehensively discuss advanced data types and their applications in tackling complex problems for seasoned data scientists as well as beginners.

The chapter discusses the challenges and options associated with data processing and model selection, particularly concerning time series data. We’ll explore different types of solutions, weigh the trade-offs, and discuss specific considerations in the field of MLOps. We will broaden our scope to include various platforms such as AWS, GCP, Hugging Face, and Apple’s CreateML. Each platform offers a unique set of tools and services that can effectively cater to different needs and preferences. By providing an unbiased comparison of these platforms and discussing their pros and cons, we aim to help you make a well-informed decision.

To get a sense of the variety of data types MLOps developers use, look at Apple’s CreateML interface in Figure 9-1. As illustrated, there are categories in Image, Video, Motion, Sound, Text, and Tables.

We’ll return to CreateML toward the end of the chapter.