book

Python Machine Learning By Example - Fourth Edition

Name: Python Machine Learning By Example - Fourth Edition
Author: Yuxi (Hayden) Liu
ISBN: 9781835085622

by Yuxi (Hayden) Liu

July 2024

Intermediate to advanced

526 pages

14h 15m

English

Packt Publishing

Read now

Unlock full access

Contributors
Join our book’s Discord space
Preface
Who this book is forWhat this book coversTo get the most out of this bookGet in touchMaking the Most Out of This Book – Get to Know Your Free BenefitsUnlock Your Book’s Exclusive BenefitsHow to unlock these benefits in three easy stepsNeed help?
Getting Started with Machine Learning and Python
An introduction to machine learningUnderstanding why we need machine learningDifferentiating between machine learning and automationMachine learning applicationsKnowing the prerequisitesGetting started with three types of machine learningA brief history of the development of machine learning algorithmsDigging into the core of machine learningGeneralizing with dataOverfitting, underfitting, and the bias-variance trade-offOverfittingUnderfittingThe bias-variance trade-offAvoiding overfitting with cross-validationAvoiding overfitting with regularizationAvoiding overfitting with feature selection and dimensionality reductionData preprocessing and feature engineeringPreprocessing and explorationDealing with missing valuesLabel encodingOne-hot encodingDense embeddingScalingFeature engineeringPolynomial transformationBinningCombining modelsVoting and averagingBaggingBoostingStackingInstalling software and setting upSetting up Python and environmentsInstalling the main Python packagesNumPySciPypandasscikit-learnTensorFlowPyTorchSummaryExercisesJoin our book’s Discord space
Building a Movie Recommendation Engine with Naïve Bayes
Getting started with classificationBinary classificationMulticlass classificationMulti-label classificationExploring Naïve BayesBayes’ theorem by exampleThe mechanics of Naïve BayesImplementing Naïve BayesImplementing Naïve Bayes from scratchImplementing Naïve Bayes with scikit-learnBuilding a movie recommender with Naïve BayesPreparing the dataTraining a Naïve Bayes modelEvaluating classification performanceTuning models with cross-validationSummaryExercisesReferences
Predicting Online Ad Click-Through with Tree-Based Algorithms
A brief overview of ad click-through predictionGetting started with two types of data – numerical and categoricalExploring a decision tree from the root to the leavesConstructing a decision treeThe metrics for measuring a splitGini ImpurityInformation GainImplementing a decision tree from scratchImplementing a decision tree with scikit-learnPredicting ad click-through with a decision treeEnsembling decision trees – random forestsEnsembling decision trees – gradient-boosted treesSummaryExercisesJoin our book’s Discord space
Predicting Online Ad Click-Through with Logistic Regression
Converting categorical features to numerical – one-hot encoding and ordinal encodingClassifying data with logistic regressionGetting started with the logistic functionJumping from the logistic function to logistic regressionTraining a logistic regression modelTraining a logistic regression model using gradient descentPredicting ad click-through with logistic regression using gradient descentTraining a logistic regression model using stochastic gradient descent (SGD)Training a logistic regression model with regularizationFeature selection using L1 regularizationFeature selection using random forestTraining on large datasets with online learningHandling multiclass classificationImplementing logistic regression using TensorFlowSummaryExercises
Predicting Stock Prices with Regression Algorithms
What is regression?Mining stock price dataA brief overview of the stock market and stock pricesGetting started with feature engineeringAcquiring data and generating featuresEstimating with linear regressionHow does linear regression work?Implementing linear regression from scratchImplementing linear regression with scikit-learnImplementing linear regression with TensorFlowEstimating with decision tree regressionTransitioning from classification trees to regression treesImplementing decision tree regressionImplementing a regression forestEvaluating regression performancePredicting stock prices with the three regression algorithmsSummaryExercisesJoin our book’s Discord space
Predicting Stock Prices with Artificial Neural Networks
Demystifying neural networksStarting with a single-layer neural networkLayers in neural networksActivation functionsBackpropagationAdding more layers to a neural network: DLBuilding neural networksImplementing neural networks from scratchImplementing neural networks with scikit-learnImplementing neural networks with TensorFlowImplementing neural networks with PyTorchPicking the right activation functionsPreventing overfitting in neural networksDropoutEarly stoppingPredicting stock prices with neural networksTraining a simple neural networkFine-tuning the neural networkSummaryExercises
Mining the 20 Newsgroups Dataset with Text Analysis Techniques
How computers understand language – NLPWhat is NLP?The history of NLPNLP applicationsTouring popular NLP libraries and picking up NLP basicsInstalling famous NLP librariesCorporaTokenizationPoS taggingNERStemming and lemmatizationSemantics and topic modelingGetting the newsgroups dataExploring the newsgroups dataThinking about features for text dataCounting the occurrence of each word tokenText preprocessingDropping stop wordsReducing inflectional and derivational forms of wordsVisualizing the newsgroups data with t-SNEWhat is dimensionality reduction?t-SNE for dimensionality reductionRepresenting words with dense vectors – word embeddingBuilding embedding models using shallow neural networksUtilizing pre-trained embedding modelsSummaryExercisesJoin our book’s Discord space
Discovering Underlying Topics in the Newsgroups Dataset with Clustering and Topic Modeling
Learning without guidance – unsupervised learningGetting started with k-means clusteringHow does k-means clustering work?Implementing k-means from scratchImplementing k-means with scikit-learnChoosing the value of kClustering newsgroups datasetClustering newsgroups data using k-meansDescribing the clusters using GPTDiscovering underlying topics in newsgroupsTopic modeling using NMFTopic modeling using LDASummaryExercises

Recognizing Faces with Support Vector Machine
Finding the separating boundary with SVMScenario 1 – identifying a separating hyperplaneScenario 2 – determining the optimal hyperplaneScenario 3 – handling outliersImplementing SVMScenario 4 – dealing with more than two classesOne-vs-restOne-vs-oneMulticlass cases in scikit-learnScenario 5 – solving linearly non-separable problems with kernelsChoosing between linear and RBF kernelsClassifying face images with SVMExploring the face image datasetBuilding an SVM-based image classifierBoosting image classification performance with PCAEstimating with support vector regressionImplementing SVRSummaryExercisesJoin our book’s Discord space
Machine Learning Best Practices
Machine learning solution workflowBest practices in the data preparation stageBest practice 1 – Completely understanding the project goalBest practice 2 – Collecting all fields that are relevantBest practice 3 – Maintaining the consistency and normalization of field valuesBest practice 4 – Dealing with missing dataBest practice 5 – Storing large-scale dataBest practices in the training set generation stageBest practice 6 – Identifying categorical features with numerical valuesBest practice 7 – Deciding whether to encode categorical featuresBest practice 8 – Deciding whether to select features and, if so, how to do soBest practice 9 – Deciding whether to reduce dimensionality and, if so, how to do soBest practice 10 – Deciding whether to rescale featuresBest practice 11 – Performing feature engineering with domain expertiseBest practice 12 – Performing feature engineering without domain expertiseBinarization and discretizationInteractionPolynomial transformationBest practice 13 – Documenting how each feature is generatedBest practice 14 – Extracting features from text datatf and tf-idfWord embeddingWord2Vec embeddingBest practices in the model training, evaluation, and selection stageBest practice 15 – Choosing the right algorithm(s) to start withNaïve BayesLogistic regressionSVMRandom forest (or decision tree)Neural networksBest practice 16 – Reducing overfittingBest practice 17 – Diagnosing overfitting and underfittingBest practice 18 – Modeling on large-scale datasetsBest practices in the deployment and monitoring stageBest practice 19 – Saving, loading, and reusing modelsSaving and restoring models using pickleSaving and restoring models in TensorFlowSaving and restoring models in PyTorchBest practice 20 – Monitoring model performanceBest practice 21 – Updating models regularlySummaryExercises
Categorizing Images of Clothing with Convolutional Neural Networks
Getting started with CNN building blocksThe convolutional layerThe non-linear layerThe pooling layerArchitecting a CNN for classificationExploring the clothing image datasetClassifying clothing images with CNNsArchitecting the CNN modelFitting the CNN modelVisualizing the convolutional filtersBoosting the CNN classifier with data augmentationFlipping for data augmentationRotation for data augmentationCropping for data augmentationImproving the clothing image classifier with data augmentationAdvancing the CNN classifier with transfer learningDevelopment of CNN architectures and pretrained modelsImproving the clothing image classifier by fine-tuning ResNetsSummaryExercisesJoin our book’s Discord space
Making Predictions with Sequences Using Recurrent Neural Networks
Introducing sequential learningLearning the RNN architecture by exampleRecurrent mechanismMany-to-one RNNsOne-to-many RNNsMany-to-many (synced) RNNsMany-to-many (unsynced) RNNsTraining an RNN modelOvercoming long-term dependencies with LSTMAnalyzing movie review sentiment with RNNsAnalyzing and preprocessing the dataBuilding a simple LSTM networkStacking multiple LSTM layersRevisiting stock price forecasting with LSTMWriting your own War and Peace with RNNsAcquiring and analyzing the training dataConstructing the training set for the RNN text generatorBuilding and training an RNN text generatorSummaryExercises
Advancing Language Understanding and Generation with the Transformer Models
Understanding self-attentionKey, value, and query representationsAttention score calculation and embedding vector generationMulti-head attentionExploring the Transformer’s architectureThe encoder-decoder structurePositional encodingLayer normalizationImproving sentiment analysis with BERT and TransformersPre-training BERTMLMNSPFine-tuning of BERTFine-tuning a pre-trained BERT model for sentiment analysisUsing the Trainer API to train Transformer modelsGenerating text using GPTPre-training of GPT and autoregressive generationWriting your own version of War and Peace with GPTSummaryExercisesJoin our book’s Discord space
Building an Image Search Engine Using CLIP: a Multimodal Approach
Introducing the CLIP modelUnderstanding the mechanism of the CLIP modelVision encoderText encoderContrastive learningExploring applications of the CLIP modelZero-shot image classificationZero-shot text classificationImage and text retrievalImage and text generationTransfer learningGetting started with the datasetObtaining the Flickr8k datasetLoading the Flickr8k datasetArchitecting the CLIP modelVision encoderText encoderProjection head for contrastive learningCLIP modelFinding images with wordsTraining a CLIP modelObtaining embeddings for images and text to identify matchesImage search using the pre-trained CLIP modelZero-shot classificationSummaryExercisesReferences
Making Decisions in Complex Environments with Reinforcement Learning
Setting up the working environmentIntroducing OpenAI Gym and GymnasiumInstalling GymnasiumIntroducing reinforcement learning with examplesElements of reinforcement learningCumulative rewardsApproaches to reinforcement learningPolicy-based approachValue-based approachSolving the FrozenLake environment with dynamic programmingSimulating the FrozenLake environmentSolving FrozenLake with the value iteration algorithmSolving FrozenLake with the policy iteration algorithmPerforming Monte Carlo learningSimulating the Blackjack environmentPerforming Monte Carlo policy evaluationPerforming on-policy Monte Carlo controlSolving the Blackjack problem with the Q-learning algorithmIntroducing the Q-learning algorithmDeveloping the Q-learning algorithmSummaryExercisesJoin our book’s Discord space
Other Books You May Enjoy
Index

Content preview from Python Machine Learning By Example - Fourth Edition

13 Advancing Language Understanding and Generation with the Transformer Models

In the previous chapter, we focused on RNNs and used them to deal with sequence learning tasks. However, RNNs may easily suffer from the vanishing gradient problem. In this chapter, we will explore the Transformer neural network architecture, which is designed for sequence-to-sequence tasks and is particularly well suited for Natural Language Processing (NLP). The key innovation is the self-attention mechanism, allowing the model to weigh different parts of the input sequence differently, and enabling it to capture long-range dependencies more effectively than RNNs.

We will learn two cutting-edge models utilizing the Transformer architecture and delve into their practical ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781835085622

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Python Machine Learning By Example - Fourth Edition

by Yuxi (Hayden) Liu

13

Advancing Language Understanding and Generation with the Transformer Models

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.