book

Machine Learning for Imbalanced Data

Name: Machine Learning for Imbalanced Data
ISBN: 9781801070836

by Kumar Abhishek, Dr. Mounir Abdelaziz

November 2023

Intermediate to advanced

344 pages

9h 9m

English

Packt Publishing

Read now

Unlock full access

Machine Learning for Imbalanced Data
ContributorsAbout the authorsOther contributorAbout the reviewers
Preface
Who this book is forWhat this book covers📌 Usage of techniques – In production tipsTo get the most out of this bookDownload the example code filesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Chapter 1: Introduction to Data Imbalance in Machine Learning
Technical requirementsIntroduction to imbalanced datasetsMachine learning 101What happens during model training?Types of dataset and splitsCross-validationCommon evaluation metricsConfusion matrixROCPrecision-Recall curveRelation between the ROC curve and PR curveChallenges and considerations when dealing with imbalanced dataWhen can we have an imbalance in datasets?Why can imbalanced data be a challenge?When to not worry about data imbalanceIntroduction to the imbalanced-learn libraryGeneral rules to followSummaryQuestionsReferences
Chapter 2: Oversampling Methods
Technical requirementsWhat is oversampling?Random oversamplingProblems with random oversamplingSMOTEHow SMOTE worksProblems with SMOTESMOTE variantsBorderline-SMOTEADASYNWorking of ADASYNCategorical features and SMOTE variants (SMOTE-NC and SMOTEN)Model performance comparison of various oversampling methodsGuidance for using various oversampling techniquesWhen to avoid oversamplingOversampling in multi-class classificationSummaryExercisesReferences
Chapter 3: Undersampling Methods
Technical requirementsIntroducing undersamplingWhen to avoid undersampling the majority classFixed versus cleaning undersamplingUndersampling approachesRemoving examples uniformlyRandom UnderSamplingClusterCentroidsStrategies for removing noisy observationsENN, RENN, and AllKNNTomek linksNeighborhood Cleaning RuleInstance hardness thresholdStrategies for removing easy observationsCondensed Nearest NeighborsOne-sided selectionCombining undersampling and oversamplingModel performance comparisonSummaryExercisesReferences
Chapter 4: Ensemble Methods
Technical requirementsBagging techniques for imbalanced dataUnderBaggingOverBaggingSMOTEBaggingComparative performance of bagging methodsBoosting techniques for imbalanced dataAdaBoostRUSBoost, SMOTEBoost, and RAMOBoostEnsemble of ensemblesEasyEnsembleComparative performance of boosting methodsModel performance comparisonSummaryQuestionsReferences
Chapter 5: Cost-Sensitive Learning
Technical requirementsThe concept of Cost-Sensitive LearningCosts and cost functionsTypes of cost-sensitive learningDifference between CSL and resamplingProblems with rebalancing techniquesUnderstanding costs in practiceCost-Sensitive Learning for logistic regressionCost-Sensitive Learning for decision treesCost-Sensitive Learning using scikit-learn and XGBoost modelsMetaCost – making any classification model cost-sensitiveThreshold adjustmentMethods for threshold tuningSummaryQuestionsReferences
Chapter 6: Data Imbalance in Deep Learning
Technical requirementsA brief introduction to deep learningNeural networksPerceptronActivation functionsLayersFeedforward neural networksTraining neural networksThe effect of the learning rate on data imbalanceImage processing using Convolutional Neural NetworksText analysis using Natural Language ProcessingData imbalance in deep learningThe impact of data imbalance on deep learning modelsOverview of deep learning techniques to handle data imbalanceMulti-label classificationSummaryQuestionsReferences
Chapter 7: Data-Level Deep Learning Methods
Technical requirementsPreparing the dataCreating the training loopSampling techniques for deep learning modelsRandom oversamplingDynamic samplingData augmentation techniques for visionData-level techniques for text classificationDataset and baseline modelDocument-level augmentationCharacter and word-level augmentationDiscussion of other data-level deep learning methods and their key ideasTwo-phase learningExpansive Over-SamplingUsing generative models for oversamplingDeepSMOTENeural style transferSummaryQuestionsReferences
Chapter 8: Algorithm-Level Deep Learning Techniques
Technical requirementsMotivation for algorithm-level techniquesWeighting techniquesUsing PyTorch’s weight parameterHandling textual dataDeferred re-weighting – a minor variant of the class weighting techniqueExplicit loss function modificationFocal lossClass-balanced lossClass-dependent temperature LossClass-wise difficulty-balanced lossDiscussing other algorithm-based techniquesRegularization techniquesSiamese networksDeeper neural networksThreshold adjustmentSummaryQuestionsReferences

Chapter 9: Hybrid Deep Learning Methods
Technical requirementsUsing graph machine learning for imbalanced dataUnderstanding graphsGraph machine learningDealing with imbalanced dataCase study – the performance of XGBoost, MLP, and a GCN on an imbalanced datasetHard example miningOnline Hard Example MiningMinority class incremental rectificationUtilizing the hard sample mining technique in minority class incremental rectificationSummaryQuestionsReferences
Chapter 10: Model Calibration
Technical requirementsIntroduction to model calibrationWhy bother with model calibrationModels with and without well-calibrated probabilitiesCalibration curves or reliability plotBrier scoreExpected Calibration ErrorThe influence of data balancing techniques on model calibrationPlotting calibration curves for a model trained on a real-world datasetModel calibration techniquesThe calibration of model scores to account for samplingPlatt’s scalingIsotonic regressionChoosing between Platt’s scaling and Isotonic regressionTemperature scalingLabel smoothingThe impact of calibration on a model’s performanceSummaryQuestionsReferences
Appendix: Machine Learning Pipeline in Production
Machine learning training pipelineInferencing (online or batch)
Assessments
Chapter 1 – Introduction to Data Imbalance in Machine LearningChapter 2 – Oversampling MethodsChapter 3 – Undersampling MethodsChapter 4 – Ensemble MethodsChapter 5 – Cost-Sensitive LearningChapter 6 – Data Imbalance in Deep LearningChapter 7 – Data-Level Deep Learning MethodsChapter 8 – Algorithm-Level Deep Learning TechniquesChapter 9 – Hybrid Deep Learning MethodsChapter 10 – Model Calibration
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from Machine Learning for Imbalanced Data

Appendix

Machine Learning Pipeline in Production

In this appendix, we will look at when and at which step we incorporate the data imbalance-handling techniques within a production machine learning pipeline. This mainly applies to supervised classification problems.

Machine learning training pipeline

A machine learning pipeline is the end-to-end process of training one or more machine learning models and then deploying them to a live environment. It may involve stages such as data collection, model training, validation, deployment, monitoring, and iterative improvement, with a focus on scalability, efficiency, and robustness.

Various steps during the offline training are shown in Figure A.1. Please note that some of the steps may not be necessary ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Machine Learning for High-Risk Applications

Publisher Resources

ISBN: 9781801070836

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design