book

Practical Machine Learning on Databricks

Name: Practical Machine Learning on Databricks
Author: Debu Sinha
ISBN: 9781801812030

by Debu Sinha

November 2023

Intermediate to advanced

244 pages

5h 45m

English

Packt Publishing

Read now

Unlock full access

Contributors
About the authorAbout the reviewers
Preface
Who this book is for?What this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchReviewsShare Your ThoughtsDownload a free PDF copy of this book
Part 1: Introduction
Chapter 1: The ML Process and Its Challenges
Understanding the typical machine learning processDiscovering the roles associated with machine learning projects in organizationsChallenges with productionizing machine learning use cases in organizationsUnderstanding the requirements of an enterprise-grade machine learning platformScalability – the growth catalystPerformance – ensuring efficiency and speedSecurity – safeguarding data and modelsGovernance – steering the machine learning life cycleReproducibility – ensuring trust and consistencyEase of use – balancing complexity and usabilityExploring Databricks and the Lakehouse architectureScalability – the growth catalystPerformance – ensuring efficiency and speedSecurity – safeguarding data and modelsGovernance – steering the machine learning life cycleReproducibility – ensuring trust and consistencyEase of use – balancing complexity and usabilitySimplifying machine learning development with the Lakehouse architectureSummaryFurther reading
Chapter 2: Overview of ML on Databricks
Technical requirementsSetting up a Databricks trial accountExploring the workspaceReposExploring clustersSingle userSharedNo isolation sharedSingle-node clustersExploring notebooksExploring dataExploring experimentsDiscovering the feature storeDiscovering the model registryLibrariesStoring librariesManaging librariesDatabricks Runtime and librariesLibrary usage modesUnity Catalog limitationsInstallation sources for librariesSummaryFurther reading
Part 2: ML Pipeline Components and Implementation
Chapter 3: Utilizing the Feature Store
Technical requirementsDiving into feature stores and the problems they solveDiscovering feature stores on the Databricks platformFeature tableOffline storeOnline storeTraining SetModel packagingRegistering your first feature table in Databricks Feature StoreSummaryFurther reading
Chapter 4: Understanding MLflow Components on Databricks
Technical requirementsOverview of MLflowMLflow TrackingMLflow ModelsMLflow Model RegistryExample code showing how to track ML model training in DatabricksSummary
Chapter 5: Create a Baseline Model Using Databricks AutoML
Technical requirementsUnderstanding the need for AutoMLUnderstanding AutoML in DatabricksSampling large datasetsImbalance data detectionSplitting data into train/validation/test setsEnhancing semantic type detectionShapley value (SHAP) for model explainabilityFeature Store integrationRunning AutoML on our churn prediction datasetSummaryFurther reading
Part 3: ML Governance and Deployment

Chapter 6: Model Versioning and Webhooks
Technical requirementsUnderstanding the need for the Model RegistryRegistering your candidate model to the Model Registry and managing accessDiving into the webhooks support in the Model RegistrySummaryFurther reading
Chapter 7: Model Deployment Approaches
Technical requirementsUnderstanding ML deployments and paradigmsDeploying ML models for batch and streaming inferenceBatch inference on DatabricksStreaming inference on DatabricksDeploying ML models for real-time inferenceIn-depth analysis of the constraints and capabilities of Databricks Model ServingIncorporating custom Python libraries into MLflow models for Databricks deploymentDeploying custom models with MLflow and Model ServingPackaging dependencies with MLflow modelsSummaryFurther reading
Chapter 8: Automating ML Workflows Using Databricks Jobs
Technical requirementsUnderstanding Databricks WorkflowsUtilizing Databricks Workflows with Jobs to automate model training and testingSummaryFurther reading
Chapter 9: Model Drift Detection and Retraining
Technical requirementsThe motivation behind model monitoringIntroduction to model driftIntroduction to Statistical Drift Techniques for drift detectionHypothesis testingStatistical tests and measurements for numeric featuresStatistical tests and measurements for categorical featuresStatistical tests and measurements on modelsImplementing drift detection on DatabricksSummary
Chapter 10: Using CI/CD to Automate Model Retraining and Redeployment
Introduction to MLOpsDelta Lake – more than just a data lakeComprehensive model management with Databricks MLflowIntegrating DevOps and MLOps for robust ML pipelines with DatabricksFundamentals of MLOps and deployment patternsNavigating environment isolation in Databricks – multiple strategies for MLOpsUnderstanding ML deployment patternsThe deploy models approachThe deploy code approachSummaryFurther reading
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Overview

"Practical Machine Learning on Databricks" is your go-to guide for mastering machine learning workflows specifically tailored for Databricks. With this book, you'll discover how to transition from traditional DIY systems to scalable, efficient ML pipelines on Databricks, taking full advantage of its AutoML and MLflow capabilities.

What this Book will help me do

Understand and utilize Databricks for seamless ML model transitions.
Implement AutoML workflows to accelerate experiment setups.
Leverage MLflow for robust model tracking and governance.
Harness the Databricks Feature Store for efficient data preparation.
Automate model processes including retraining through Databricks workflows.

Author(s)

Debu Sinha has extensive experience in machine learning and data engineering, specializing in platforms like Databricks. Leveraging years of expertise, Debu provides insightful, hands-on instructions on navigating and mastering modern ML platforms.

Who is it for?

This book is ideal for experienced data scientists and developers familiar with Python, statistics, and the ML lifecycle, who are transitioning from custom setups to Databricks-based workflows. If you're looking to enhance your deployment pipelines and efficiently manage ML projects on Databricks, this book is for you.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781801812030

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills