book

Python Machine Learning - Third Edition

Name: Python Machine Learning - Third Edition
ISBN: 9781789955750

by Sebastian Raschka, Vahid Mirjalili

December 2019

Beginner to intermediate

772 pages

19h 20m

English

Packt Publishing

Read now

Unlock full access

Preface
Get started with machine learningPractice and theoryWhy Python?Explore the machine learning fieldWho this book is forWhat this book coversWhat you need for this bookTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchReviews
Giving Computers the Ability to Learn from Data
Building intelligent machines to transform data into knowledgeThe three different types of machine learningMaking predictions about the future with supervised learningClassification for predicting class labelsRegression for predicting continuous outcomesSolving interactive problems with reinforcement learningDiscovering hidden structures with unsupervised learningFinding subgroups with clusteringDimensionality reduction for data compressionIntroduction to the basic terminology and notationsNotation and conventions used in this bookMachine learning terminologyA roadmap for building machine learning systemsPreprocessing – getting data into shapeTraining and selecting a predictive modelEvaluating models and predicting unseen data instancesUsing Python for machine learningInstalling Python and packages from the Python Package IndexUsing the Anaconda Python distribution and package managerPackages for scientific computing, data science, and machine learningSummary
Training Simple Machine Learning Algorithms for Classification
Artificial neurons – a brief glimpse into the early history of machine learningThe formal definition of an artificial neuronThe perceptron learning ruleImplementing a perceptron learning algorithm in PythonAn object-oriented perceptron APITraining a perceptron model on the Iris datasetAdaptive linear neurons and the convergence of learningMinimizing cost functions with gradient descentImplementing Adaline in PythonImproving gradient descent through feature scalingLarge-scale machine learning and stochastic gradient descentSummary
A Tour of Machine Learning Classifiers Using scikit-learn
Choosing a classification algorithmFirst steps with scikit-learn – training a perceptronModeling class probabilities via logistic regressionLogistic regression and conditional probabilitiesLearning the weights of the logistic cost functionConverting an Adaline implementation into an algorithm for logistic regressionTraining a logistic regression model with scikit-learnTackling overfitting via regularizationMaximum margin classification with support vector machinesMaximum margin intuitionDealing with a nonlinearly separable case using slack variablesAlternative implementations in scikit-learnSolving nonlinear problems using a kernel SVMKernel methods for linearly inseparable dataUsing the kernel trick to find separating hyperplanes in a high-dimensional spaceDecision tree learningMaximizing IG – getting the most bang for your buckBuilding a decision treeCombining multiple decision trees via random forestsK-nearest neighbors – a lazy learning algorithmSummary
Building Good Training Datasets – Data Preprocessing
Dealing with missing dataIdentifying missing values in tabular dataEliminating training examples or features with missing valuesImputing missing valuesUnderstanding the scikit-learn estimator APIHandling categorical dataCategorical data encoding with pandasMapping ordinal featuresEncoding class labelsPerforming one-hot encoding on nominal featuresPartitioning a dataset into separate training and test datasetsBringing features onto the same scaleSelecting meaningful featuresL1 and L2 regularization as penalties against model complexityA geometric interpretation of L2 regularizationSparse solutions with L1 regularizationSequential feature selection algorithmsAssessing feature importance with random forestsSummary
Compressing Data via Dimensionality Reduction
Unsupervised dimensionality reduction via principal component analysisThe main steps behind principal component analysisExtracting the principal components step by stepTotal and explained varianceFeature transformationPrincipal component analysis in scikit-learnSupervised data compression via linear discriminant analysisPrincipal component analysis versus linear discriminant analysisThe inner workings of linear discriminant analysisComputing the scatter matricesSelecting linear discriminants for the new feature subspaceProjecting examples onto the new feature spaceLDA via scikit-learnUsing kernel principal component analysis for nonlinear mappingsKernel functions and the kernel trickImplementing a kernel principal component analysis in PythonExample 1 – separating half-moon shapesExample 2 – separating concentric circlesProjecting new data pointsKernel principal component analysis in scikit-learnSummary
Learning Best Practices for Model Evaluation and Hyperparameter Tuning
Streamlining workflows with pipelinesLoading the Breast Cancer Wisconsin datasetCombining transformers and estimators in a pipelineUsing k-fold cross-validation to assess model performanceThe holdout methodK-fold cross-validationDebugging algorithms with learning and validation curvesDiagnosing bias and variance problems with learning curvesAddressing over- and underfitting with validation curvesFine-tuning machine learning models via grid searchTuning hyperparameters via grid searchAlgorithm selection with nested cross-validationLooking at different performance evaluation metricsReading a confusion matrixOptimizing the precision and recall of a classification modelPlotting a receiver operating characteristicScoring metrics for multiclass classificationDealing with class imbalanceSummary
Combining Different Models for Ensemble Learning
Learning with ensemblesCombining classifiers via majority voteImplementing a simple majority vote classifierUsing the majority voting principle to make predictionsEvaluating and tuning the ensemble classifierBagging – building an ensemble of classifiers from bootstrap samplesBagging in a nutshellApplying bagging to classify examples in the Wine datasetLeveraging weak learners via adaptive boostingHow boosting worksApplying AdaBoost using scikit-learnSummary
Applying Machine Learning to Sentiment Analysis
Preparing the IMDb movie review data for text processingObtaining the movie review datasetPreprocessing the movie dataset into a more convenient formatIntroducing the bag-of-words modelTransforming words into feature vectorsAssessing word relevancy via term frequency-inverse document frequencyCleaning text dataProcessing documents into tokensTraining a logistic regression model for document classificationWorking with bigger data – online algorithms and out-of-core learningTopic modeling with Latent Dirichlet AllocationDecomposing text documents with LDALDA with scikit-learnSummary
Embedding a Machine Learning Model into a Web Application
Serializing fitted scikit-learn estimatorsSetting up an SQLite database for data storageDeveloping a web application with FlaskOur first Flask web applicationForm validation and renderingSetting up the directory structureImplementing a macro using the Jinja2 templating engineAdding style via CSSCreating the result pageTurning the movie review classifier into a web applicationFiles and folders – looking at the directory treeImplementing the main application as app.pySetting up the review formCreating a results page templateDeploying the web application to a public serverCreating a PythonAnywhere accountUploading the movie classifier applicationUpdating the movie classifierSummary

Predicting Continuous Target Variables with Regression Analysis
Introducing linear regressionSimple linear regressionMultiple linear regressionExploring the Housing datasetLoading the Housing dataset into a data frameVisualizing the important characteristics of a datasetLooking at relationships using a correlation matrixImplementing an ordinary least squares linear regression modelSolving regression for regression parameters with gradient descentEstimating the coefficient of a regression model via scikit-learnFitting a robust regression model using RANSACEvaluating the performance of linear regression modelsUsing regularized methods for regressionTurning a linear regression model into a curve – polynomial regressionAdding polynomial terms using scikit-learnModeling nonlinear relationships in the Housing datasetDealing with nonlinear relationships using random forestsDecision tree regressionRandom forest regressionSummary
Working with Unlabeled Data – Clustering Analysis
Grouping objects by similarity using k-meansK-means clustering using scikit-learnA smarter way of placing the initial cluster centroids using k-means++Hard versus soft clusteringUsing the elbow method to find the optimal number of clustersQuantifying the quality of clustering via silhouette plotsOrganizing clusters as a hierarchical treeGrouping clusters in bottom-up fashionPerforming hierarchical clustering on a distance matrixAttaching dendrograms to a heat mapApplying agglomerative clustering via scikit-learnLocating regions of high density via DBSCANSummary
Implementing a Multilayer Artificial Neural Network from Scratch
Modeling complex functions with artificial neural networksSingle-layer neural network recapIntroducing the multilayer neural network architectureActivating a neural network via forward propagationClassifying handwritten digitsObtaining and preparing the MNIST datasetImplementing a multilayer perceptronTraining an artificial neural networkComputing the logistic cost functionDeveloping your understanding of backpropagationTraining neural networks via backpropagationAbout the convergence in neural networksA few last words about the neural network implementationSummary
Parallelizing Neural Network Training with TensorFlow
TensorFlow and training performancePerformance challengesWhat is TensorFlow?How we will learn TensorFlowFirst steps with TensorFlowInstalling TensorFlowCreating tensors in TensorFlowManipulating the data type and shape of a tensorApplying mathematical operations to tensorsSplit, stack, and concatenate tensorsBuilding input pipelines using tf.data – the TensorFlow Dataset APICreating a TensorFlow Dataset from existing tensorsCombining two tensors into a joint datasetShuffle, batch, and repeatCreating a dataset from files on your local storage diskFetching available datasets from the tensorflow_datasets libraryBuilding an NN model in TensorFlowThe TensorFlow Keras API (tf.keras)Building a linear regression modelModel training via the .compile() and .fit() methodsBuilding a multilayer perceptron for classifying flowers in the Iris datasetEvaluating the trained model on the test datasetSaving and reloading the trained modelChoosing activation functions for multilayer neural networksLogistic function recapEstimating class probabilities in multiclass classification via the softmax functionBroadening the output spectrum using a hyperbolic tangentRectified linear unit activationSummary
Going Deeper – The Mechanics of TensorFlow
The key features of TensorFlowTensorFlow's computation graphs: migrating to TensorFlow v2Understanding computation graphsCreating a graph in TensorFlow v1.xMigrating a graph to TensorFlow v2Loading input data into a model: TensorFlow v1.x styleLoading input data into a model: TensorFlow v2 styleImproving computational performance with function decoratorsTensorFlow Variable objects for storing and updating model parametersComputing gradients via automatic differentiation and GradientTapeComputing the gradients of the loss with respect to trainable variablesComputing gradients with respect to non-trainable tensorsKeeping resources for multiple gradient computationsSimplifying implementations of common architectures via the Keras APISolving an XOR classification problemMaking model building more flexible with Keras' functional APIImplementing models based on Keras' Model classWriting custom Keras layersTensorFlow EstimatorsWorking with feature columnsMachine learning with pre-made EstimatorsUsing Estimators for MNIST handwritten digit classificationCreating a custom Estimator from an existing Keras modelSummary
Classifying Images with Deep Convolutional Neural Networks
The building blocks of CNNsUnderstanding CNNs and feature hierarchiesPerforming discrete convolutionsDiscrete convolutions in one dimensionPadding inputs to control the size of the output feature mapsDetermining the size of the convolution outputPerforming a discrete convolution in 2DSubsampling layersPutting everything together – implementing a CNNWorking with multiple input or color channelsRegularizing an NN with dropoutLoss functions for classificationImplementing a deep CNN using TensorFlowThe multilayer CNN architectureLoading and preprocessing the dataImplementing a CNN using the TensorFlow Keras APIConfiguring CNN layers in KerasConstructing a CNN in KerasGender classification from face images using a CNNLoading the CelebA datasetImage transformation and data augmentationTraining a CNN gender classifierSummary
Modeling Sequential Data Using Recurrent Neural Networks
Introducing sequential dataModeling sequential data – order mattersRepresenting sequencesThe different categories of sequence modelingRNNs for modeling sequencesUnderstanding the RNN looping mechanismComputing activations in an RNNHidden-recurrence versus output-recurrenceThe challenges of learning long-range interactionsLong short-term memory cellsImplementing RNNs for sequence modeling in TensorFlowProject one – predicting the sentiment of IMDb movie reviewsPreparing the movie review dataEmbedding layers for sentence encodingBuilding an RNN modelBuilding an RNN model for the sentiment analysis taskProject two – character-level language modeling in TensorFlowPreprocessing the datasetBuilding a character-level RNN modelEvaluation phase – generating new text passagesUnderstanding language with the Transformer modelUnderstanding the self-attention mechanismA basic version of self-attentionParameterizing the self-attention mechanism with query, key, and value weightsMulti-head attention and the Transformer blockSummary
Generative Adversarial Networks for Synthesizing New Data
Introducing generative adversarial networksStarting with autoencodersGenerative models for synthesizing new dataGenerating new samples with GANsUnderstanding the loss functions of the generator and discriminator networks in a GAN modelImplementing a GAN from scratchTraining GAN models on Google ColabImplementing the generator and the discriminator networksDefining the training datasetTraining the GAN modelImproving the quality of synthesized images using a convolutional and Wasserstein GANTransposed convolutionBatch normalizationImplementing the generator and discriminatorDissimilarity measures between two distributionsUsing EM distance in practice for GANsGradient penaltyImplementing WGAN-GP to train the DCGAN modelMode collapseOther GAN applicationsSummary
Reinforcement Learning for Decision Making in Complex Environments
Introduction – learning from experienceUnderstanding reinforcement learningDefining the agent-environment interface of a reinforcement learning systemThe theoretical foundations of RLMarkov decision processesThe mathematical formulation of Markov decision processesVisualization of a Markov processEpisodic versus continuing tasksRL terminology: return, policy, and value functionThe returnPolicyValue functionDynamic programming using the Bellman equationReinforcement learning algorithmsDynamic programmingPolicy evaluation – predicting the value function with dynamic programmingImproving the policy using the estimated value functionPolicy iterationValue iterationReinforcement learning with Monte CarloState-value function estimation using MCAction-value function estimation using MCFinding an optimal policy using MC controlPolicy improvement – computing the greedy policy from the action-value functionTemporal difference learningTD predictionOn-policy TD control (SARSA)Off-policy TD control (Q-learning)Implementing our first RL algorithmIntroducing the OpenAI Gym toolkitWorking with the existing environments in OpenAI GymA grid world exampleImplementing the grid world environment in OpenAI GymSolving the grid world problem with Q-learningImplementing the Q-learning algorithmA glance at deep Q-learningTraining a DQN model according to the Q-learning algorithmImplementing a deep Q-learning algorithmChapter and book summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index

Content preview from Python Machine Learning - Third Edition

12 Implementing a Multilayer Artificial Neural Network from Scratch

As you may know, deep learning is getting a lot of attention from the press and is without any doubt the hottest topic in the machine learning field. Deep learning can be understood as a subfield of machine learning that is concerned with training artificial neural networks (NNs) with many layers efficiently. In this chapter, you will learn the basic concepts of artificial NNs so that you are well equipped for the following chapters, which will introduce advanced Python-based deep learning libraries and deep neural network (DNN) architectures that are particularly well suited for image and text analyses.

The topics that we will cover in this chapter are as follows:

Gaining ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Python Machine Learning, Second Edition - Second Edition

Publisher Resources

ISBN: 9781789955750

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Python Machine Learning - Third Edition

by Sebastian Raschka, Vahid Mirjalili

12

Implementing a Multilayer Artificial Neural Network from Scratch

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.