book

Scaling Machine Learning with Spark

Name: Scaling Machine Learning with Spark
Author: Adi Polak
ISBN: 9781098106829

by Adi Polak

March 2023

Intermediate to advanced

291 pages

8h 54m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Preface
Who Should Read This Book?Do You Need Distributed Machine Learning?Navigating This BookWhat Is Not CoveredThe Environment and ToolsThe ToolsThe DatasetsConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
1. Distributed Machine Learning Terminology and Concepts
The Stages of the Machine Learning WorkflowTools and Technologies in the Machine Learning PipelineDistributed Computing ModelsGeneral-Purpose ModelsDedicated Distributed Computing ModelsIntroduction to Distributed Systems ArchitectureCentralized Versus Decentralized SystemsInteraction ModelsCommunication in a Distributed SettingIntroduction to Ensemble MethodsHigh Versus Low BiasTypes of Ensemble MethodsDistributed Training TopologiesThe Challenges of Distributed Machine Learning SystemsPerformanceResource ManagementFault TolerancePrivacyPortabilitySetting Up Your Local EnvironmentChapters 2–6 Tutorials EnvironmentChapters 7–10 Tutorials EnvironmentSummary
2. Introduction to Spark and PySpark
Apache Spark ArchitectureIntro to PySparkApache Spark BasicsSoftware ArchitecturePySpark and Functional ProgrammingExecuting PySpark Codepandas DataFrames Versus Spark DataFramesScikit-Learn Versus MLlibSummary
3. Managing the Machine Learning Experiment Lifecycle with MLflow
Machine Learning Lifecycle Management RequirementsWhat Is MLflow?Software Components of the MLflow PlatformUsers of the MLflow PlatformMLflow ComponentsMLflow TrackingMLflow ProjectsMLflow ModelsMLflow Model RegistryUsing MLflow at ScaleSummary
4. Data Ingestion, Preprocessing, and Descriptive Statistics
Data Ingestion with SparkWorking with ImagesWorking with Tabular DataPreprocessing DataPreprocessing Versus ProcessingWhy Preprocess the Data?Data StructuresMLlib Data TypesPreprocessing with MLlib TransformersPreprocessing Image DataSave the Data and Avoid the Small Files ProblemDescriptive Statistics: Getting a Feel for the DataCalculating StatisticsDescriptive Statistics with Spark SummarizerData SkewnessCorrelationSummary
5. Feature Engineering
Features and Their Impact on ModelsMLlib Featurization ToolsExtractorsSelectorsExample: Word2VecThe Image Featurization ProcessUnderstanding Image ManipulationExtracting Features with Spark APIsThe Text Featurization ProcessBag-of-WordsTF-IDFN-GramAdditional TechniquesEnriching the DatasetSummary
6. Training Models with Spark MLlib
AlgorithmsSupervised Machine LearningClassificationRegressionUnsupervised Machine LearningFrequent Pattern MiningClusteringEvaluatingSupervised EvaluatorsUnsupervised EvaluatorsHyperparameters and Tuning ExperimentsBuilding a Parameter GridSplitting the Data into Training and Test SetsCross-Validation: A Better Way to Test Your ModelsMachine Learning PipelinesConstructing a PipelineHow Does Splitting Work with the Pipeline API?PersistenceSummary
7. Bridging Spark and Deep Learning Frameworks
The Two Clusters ApproachImplementing a Dedicated Data Access LayerFeatures of a DALSelecting a DALWhat Is Petastorm?SparkDatasetConverterPetastorm as a Parquet StoreProject HydrogenBarrier Execution ModeAccelerator-Aware SchedulingA Brief Introduction to the Horovod Estimator APISummary
8. TensorFlow Distributed Machine Learning Approach
A Quick Overview of TensorFlowWhat Is a Neural Network?TensorFlow Cluster Process Roles and ResponsibilitiesLoading Parquet Data into a TensorFlow DatasetAn Inside Look at TensorFlow’s Distributed Machine Learning StrategiesParameterServerStrategyCentralStorageStrategy: One Machine, Multiple ProcessorsMirroredStrategy: One Machine, Multiple Processors, Local CopyMultiWorkerMirroredStrategy: Multiple Machines, SynchronousTPUStrategyWhat Things Change When You Switch Strategies?Training APIsKeras APICustom Training LoopEstimator APIPutting It All TogetherTroubleshootingSummary
9. PyTorch Distributed Machine Learning Approach
A Quick Overview of PyTorch BasicsComputation GraphPyTorch Mechanics and ConceptsPyTorch Distributed Strategies for Training ModelsIntroduction to PyTorch’s Distributed ApproachDistributed Data-Parallel TrainingRPC-Based Distributed TrainingCommunication Topologies in PyTorch (c10d)What Can We Do with PyTorch’s Low-Level APIs?Loading Data with PyTorch and PetastormTroubleshooting Guidance for Working with Petastorm and Distributed PyTorchThe Enigma of Mismatched Data TypesThe Mystery of Straggling WorkersHow Does PyTorch Differ from TensorFlow?Summary

10. Deployment Patterns for Machine Learning Models
Deployment PatternsPattern 1: Batch PredictionPattern 2: Model-in-ServicePattern 3: Model-as-a-ServiceDetermining Which Pattern to UseProduction Software RequirementsMonitoring Machine Learning Models in ProductionData DriftModel Drift, Concept DriftDistributional Domain Shift (the Long Tail)What Metrics Should I Monitor in Production?How Do I Measure Changes Using My Monitoring System?What It Looks Like in ProductionThe Production Feedback LoopDeploying with MLlibProduction Machine Learning Pipelines with Structured StreamingDeploying with MLflowDefining an MLflow WrapperDeploying the Model as a MicroserviceLoading the Model as a Spark UDFHow to Develop Your System IterativelySummary
Index
About the Author

Content preview from Scaling Machine Learning with Spark

About the Author

Adi Polak is an open-source technologist who believes in communities and education and their ability to positively impact the world around us. She is passionate about building a better world through open collaboration and technological innovation. As a seasoned engineer and vice president of developer experience at Treeverse, Adi shapes the future of data and machine learning technologies for hands-on builders. She serves on multiple program committees and acts as an advisor for conferences like Data & AI Summit by Databricks, Current by Confluent, and Scale by the Bay, among others. Adi previously served as a senior manager for Azure at Microsoft, where she helped build advanced analytics systems and modern data architectures. Adi gained experience in machine learning by conducting research for IBM, Deutsche Telekom, and other Fortune 500 companies.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781098106812Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Scaling Machine Learning with Spark

by Adi Polak

About the Author

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.