book

Optimizing AI and Machine Learning Solutions

Name: Optimizing AI and Machine Learning Solutions
Author: Mirza Rahim Baig
ISBN: 9789355519818

by Mirza Rahim Baig

August 2024

392 pages

10h 23m

English

De Gruyter

Read now

Unlock full access

Cover Page
Title Page
Copyright Page
Dedication
About the Author
About the Reviewer
Acknowledgement
Preface
Table of Contents
1. Optimizing a Machine Learning /Artificial Intelligence Solution
IntroductionStructureObjectivesCase study: Text deduplication for online fashionUnderstanding machine learningMachine learning stylesSupervised machine learningUnsupervised machine learning Reinforcement learningChoosing the ML styleChallenges in ML/AI Poor formulationInvalid/poor assumptionsData availability and hygieneRepresentative data (lack of)Model scalabilityInfeasible consumptionMisalignment with business outcomes ML / AI models vs. end-to-end solutionsCRISP-DM frameworkOptimization at each step of solution developmentBusiness understandingData understandingData preparationModel buildingEvaluationDeploymentConclusion

2. ML Problem Formulation: Setting the Right Objective
IntroductionStructureObjectivesIdentifying a Machine Learning problemChoosing the ML styleProblem 1: Fraud detectionApproach 1: Supervised classificationApproach 2: Unsupervised clusteringProblem 2: Predicting high-selling productsApproach 1: Supervised RegressionApproach 2: Supervised ClassificationProblem 3: Question de-duplication for an e-commerce websiteApproach 1: Supervised classification using Deep LearningApproach 2: Unsupervised clustering of questionsBlending ML stylesFor better accuracyFor dimensionality reductionFor better data representationChoosing the right dependent variableBusiness objective vs. ML objectiveConclusion
3. Data Collection and Pre-processing
IntroductionStructureObjectivesBuilding a machine learning solutionThe data collection processThe nature of the dataDomain-specific aspectsInfluence of the task at handThe case studyThe pre-processing processStep 1: Gather data + basic checksExercise 3.1: Loading and exploring bank marketing datasetExercise 3.2: Fixing formats and identifying missing valuesStep 2: Separate into train and test datasetsExercise 3.3: Splitting the data into train and test setsStep 3: Outlier treatmentPercentile threshold approachZ score approachBox plots/Tukey criterionExercise 3.4: Outlier detection and treatmentStep 4: Missing value treatmentStep 5: Categorical feature handlingSolving our case studyExercise 3.5: Handling remaining categorical featuresStep 6: Transformation of numeric featuresExercise 3.6: Transforming numerical variables:Case study steps reviewConclusion
4. Model Evaluation and Debugging
IntroductionStructureObjectivesAd click prediction case studyExercise 4.1: Load and prepare the dataModel evaluation considerationsExercise 4.2: Building the competing modelsModel evaluation metricsMetrics for classification problemsAccuracyConfusion matrixExercise 4.3: Confusion matrix for click prediction modelsPrecision, Recall, and F1 ScoreArea Under ROC curve (AUC)Area under PR curve Metrics for regression Model evaluation schemesExercise 4.4: Model evaluation using cross-validationModel debuggingOverfitting and underfittingOverfittingUnderfittingValidation curves and goodness of fitExercise 4.5: Validation curves for Ad click predictionHandling overfittingFeature selectionHyper-parameter controlRegularizationData augmentationOptimal training (early stopping)Conclusion
5. Imbalanced Machine Learning
IntroductionStructureObjectivesUnderstanding the business problemExercise 5.1: Understanding and staging the dataDifficulty in model evaluationExercise 5.2: Accuracy for imbalanced classesEvaluation metrics for imbalanced classesF1 scorePrecision-Recall CurveSummary of model evaluationHandling imbalanceAdjusting the cost functionExercise 5.3: Adjusting class_weightData balancing UndersamplingExercise 5.4: Random undersampling of insurance dataModel evaluation with re-balancingUsing Pipelines for evaluating re-balanced data Undersampling methods: Tomek LinksExercise 5.5: Tomek Links applied to insurance dataUndersampling methods: edited nearest neighbourExercise 5.6: Edited nearest neighbour undersamplingOversamplingSynthetic Minority Oversampling TechniqueExercise 5.7: Synthetic Data Generation with SMOTEADASYN - Adaptive SynthesisExercise 5.8: Synthetic Data Generation with ADASYNCombination of under and oversamplingExercise 5.9: Using a combination of over and undersamplingComparing the approachesConclusion
6. Hyper-parameter Tuning
IntroductionStructureObjectivesParameters and hyper-parametersImportance of hyper-parametersHyper-parameter tuningManual looping over hyper-parametersPerform your own manual hyper-parameter tuningGridSearchCV - Grid Search with Cross-ValidationUsing GridSearchCV for hyper-parameter tuningGridSearchCV with multiple hyper-parametersThe effect of each hyper-parameterLimitations of exhaustive methods (looping, GridSearchCV)RandomizedSearchCVGrid Search versus Randomized SearchChoosing a hyper-parameter search approach Choosing the ‘best’ modelTraining on the entire data and making predictionsThe complete processConclusion
7. Parameter Optimization Algorithms
IntroductionStructureObjectivesOptimization in machine learning Fundamentals of mathematical optimizationObjective/loss functionsLocal and global optimaConvex vs. non-convex loss functionsNumerical optimization for machine learningRegressionClassificationRegularizationA general solving approachGradient descentStopping conditionFaster gradient descent variantsMomentumAdaGrad optimizerRMSProp optimizerAdam optimizerChoosing the optimization approachConclusion
8. Optimizing Deep Learning Models
IntroductionStructureObjectivesThe image processing case studyThe overall modeling approachBuilding a baseline modelHyper-parameters in neural networksActivation functionLearning rateTuning the learning rateNetwork structureOptimizerOther hyper-parametersCoarse plus fine tuningPractical tips for tuning Neural NetworksBeyond hyper-parameters: other approaches to boost performanceRegularizationMore dataData augmentationEnsemblesInject Domain knowledge through features ConclusionAdditional reads
9. Optimizing Image Models
IntroductionStructureObjectivesApplications of image processingCase study: Auto image classificationFeature detection using convolutionsTraditional Convolutional Neural NetworksParameter regularizationSpatial DropoutExercise 9.1: Convolution model with dropoutBatch normalizationData augmentationExercise 9.2: Conv model with data augmentationPopular architecturesVGG design approachExercise 9.3: Creating VGG16 from scratchResNet design approachGuidelines for designing image modelsConclusion
10. Optimizing Natural Language Processing Models
IntroductionStructureObjectivesKey tasks in NLPImportance of sequence processingA text classification case studyText pre-processingText representationCount-based featuresText embeddingsEmbedding layer in KerasArchitectures for NLPRecurrent architecturesLong Short Term MemoryGated recurrent unitsBi-directional recurrent architecturesRecurrent architectures with attentionTransformer architectureTuning network hyper-parametersExercise 10.1: Hyper-parameter tuning of GRU modelUsing convolutions for NLPExercise 10.2: Using 1D convolutions and RNNsUsing pre-trained embeddingsBenefitsDrawbacksExercise- Text classification with pre-trained embeddingsNotable pre-trained embeddingsConclusion
11. Transfer Learning
IntroductionStructureObjectivesMotivation for transfer learningUsing a SOTA image classification modelWhat is transfer learning?Applications of transfer learningReady to use pre-trained modelsFine tuning modelsData representation/feature extractionBenefits of transfer learningLimitations of transfer learningSources for pre-trained modelsKeras ApplicationsTensorflow HubHugging FaceKaggleGitHubWord embeddings for textGeneral transfer learning workflowTransfer learning for imagesImageNet databaseChoosing a Transfer Learning approachFine tuning approachesFine tuning a SOTA image modelTransfer learning for textApplications of transfer learning for textPrediction using pre-trained modelsFeature extractionFine tuningHugging face for transfer learningSentiment analysisExercise 11.1: RoBERTa for tweet sentiment classificationText generationZero-shot classificationTranslationFine tuning a SOTA NLP modelExercise 11.2: Fine tuning a BERT modelConclusion
Index

Content preview from Optimizing AI and Machine Learning Solutions

CHAPTER 6Hyper-parameter Tuning

You are a data scientist in the marketing team at a bank. The bank is gearing up for its next major marketing campaign. The product to be promoted is not new, and there have been marketing campaigns in the past. Surely, the past campaign’s data can be used to improve the upcoming campaign. A good assumption is that the kind of customers who responded positively to the previous campaign will respond positively again. Therefore, you could analyze the data to identify the customer segments that are likely to respond positively. The constraints are that the output/model should be interpretable by non-technical marketers. The number of segments for the marketing team to act on should be limited. High accuracy ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9789355519818

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Optimizing AI and Machine Learning Solutions

by Mirza Rahim Baig

CHAPTER 6Hyper-parameter Tuning

Introduction

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.