book

Apache Spark for Machine Learning

Name: Apache Spark for Machine Learning
Author: Deepak Gowda
ISBN: 9781804618165

by Deepak Gowda

November 2024

Intermediate to advanced

306 pages

7h 57m

English

Packt Publishing

Read now

Unlock full access

Apache Spark for Machine Learning
Contributors
About the author
About the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownloading the example code filesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Part 1: Introduction and Fundamentals
Chapter 1: An Overview of Machine Learning Concepts
Technical requirementsUnderstanding machine learningTypes of machine learningAn introduction to Apache SparkThe background and motivation of Apache SparkChallenges with MapReduceComponents of Apache SparkUse cases and applications of Apache SparkWhy Apache Spark for machine learning?Algorithms in Apache SparkApache Spark use casesSetting up Apache SparkSummary
Chapter 2: Data Processing with Spark
Technical requirementsUnderstanding data preprocessingIngesting dataFilesystemsAmazon S3Azure Blob StorageRelational databasesNoSQL databasesAdditional data sourcesCleaning and transforming dataData cleaningData transformationAggregating dataBasic aggregationsGrouped aggregationsWindowing in SparkWhy windowing is required and its examples in SparkHow to calculate the lagData joiningTypes of data joinsSummary
Chapter 3: Feature Extraction and Transformation
Technical requirementsLearning about feature extractorsThe key aspects of feature extractorsAlgorithms for feature extractionSpark algorithms for feature extractorsCode examples for feature extractorsWorking with feature transformersThe key aspects of feature transformersUse cases and Spark algorithms for feature transformersSpark algorithms for feature transformersCode examples for feature transformersExploring feature selectorsThe key aspects of feature selectorsUse cases and Spark algorithms for feature selectorsCode examples of feature selectorsSummary
Part 2: Supervised Learning
Chapter 4: Building a Regression System
Technical requirementsLearning about regressionRegression overviewLearning regression algorithmsLinear regressionGeneralized linear regressionDecision tree regressionRandom forest regressionGradient-boosted tree regressionSurvival regressionFactorization machine regressorEvaluating the model’s performanceSelecting the evaluation metricsImproving the model’s performancePractical implementationDefining a pipeline for each regression algorithmCross-validation and hyperparameter fine-tuningSummary

Chapter 5: Building a Classification System
Technical requirementsLearning about classificationClassification overviewWhen to use the classification techniqueSome use cases of classification in machine learningDrawbacks of classification techniquesLearning about classification algorithmsLogistic regression classificationDecision tree classifierRandom forest classifierGradient-boosted tree classifierMultilayer perceptron classifierLinear SVMThe One-vs-Rest classifier (also known as One-vs-All)Naive BayesFactorization machines classifierEvaluating the model’s performanceBinary classificationMulticlass classificationAlgorithm-specific considerationsSelection tipsSelecting the evaluation metricsImplementation and validationImproving the model’s performanceCode exampleSummary
Part 3: Unsupervised Learning
Chapter 6: Building a Clustering System
Technical requirementsLearning about clusteringUnderstanding clusteringWhen to use the clustering techniqueSome use cases of clustering in machine learningPitfalls of clustering techniquesLearning clustering algorithmsK-meansLatent Dirichlet allocation (LDA)Bisecting K-meansGaussian Mixture Model (GMM)Power Iteration Clustering (PIC)Evaluating the model performanceEvaluation clustering algorithmsSelecting the evaluation metricsImproving the model performanceGeneral strategies for all modelsModel-specific strategiesSummary
Chapter 7: Building a Recommendation System
Technical requirementsAn overview of recommendation systemsUnderstanding the purpose and importance of recommendation systemsAn overview of various recommendation approachesThe need for a recommendation systemPersonalizationUser engagementBusiness growthData utilizationContent discoveryBridging supply and demandThe working mechanism of recommendation systemsContent-based recommendation systemsCollaborative filtering recommendation systemsItem-based collaborative filteringAlternating Least Squares (ALS) – the collaborative filtering algorithm in Apache SparkThe key problems and challenges in recommendation systemsCold startData sparsityImproving the quality of recommendationsEvaluating the recommendationsBuilding a recommendation system using Apache SparkSummary
Chapter 8: Mining Frequent Patterns
Technical requirementsThe basic concepts of frequent patterns and the significance of discovering patterns and rulesFrequent pattern mining applications and case studiesThe key challenges in frequent pattern miningFrequent pattern mining algorithmsFP-GrowthPrefixSpanCode examples on FPMDeveloping a model using scalable frequent pattern mining algorithmsImplementation in Apache SparkSummary
Part 4: Model Deployment
Chapter 9: Deploying a Model
Technical requirementsImportance of model deploymentPre-deployment considerationsExploring ML pipelinesCode example of building an ML pipelineModel serialization and storageModel serializationModel storageModel deployment strategiesBatch scoringConfigure the schedulerRESTful API integrationAutomating model deployment pipelineModel monitoring and managementModel performance monitoringModel updating and maintenanceScalability and performance optimizationResource managementPerformance tuningSummary
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from Apache Spark for Machine Learning

Part 1: Introduction and Fundamentals

In this part, you will embark on a journey through the foundational concepts and principles that underpin machine learning and its integration with Apache Spark. This part is designed to equip you with the essential knowledge required to navigate the more advanced topics covered later in the book.

The chapters in this part will introduce you to the core principles of machine learning, the architecture and capabilities of Apache Spark, and the techniques for feature extraction and transformation. By the end of this part, you will have a solid understanding of both the theoretical and practical aspects of these fundamental concepts.

This part contains the following chapters:

Chapter 1, An Overview of Machine ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781804618165

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Apache Spark for Machine Learning

by Deepak Gowda

Part 1: Introduction and Fundamentals

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.