book

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 3rd Edition

by Aurélien Géron

October 2022

Intermediate to advanced

864 pages

25h 31m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

The Machine Learning TsunamiMachine Learning in Your ProjectsObjective and ApproachCode ExamplesPrerequisitesRoadmapChanges Between the First and the Second EditionChanges Between the Second and the Third EditionOther ResourcesConventions Used in This BookO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Machine Learning?Why Use Machine Learning?Examples of ApplicationsTypes of Machine Learning SystemsTraining SupervisionBatch Versus Online LearningInstance-Based Versus Model-Based LearningMain Challenges of Machine LearningInsufficient Quantity of Training DataNonrepresentative Training DataPoor-Quality DataIrrelevant FeaturesOverfitting the Training DataUnderfitting the Training DataStepping BackTesting and ValidatingHyperparameter Tuning and Model SelectionData MismatchExercises
Working with Real DataLook at the Big PictureFrame the ProblemSelect a Performance MeasureCheck the AssumptionsGet the DataRunning the Code Examples Using Google ColabSaving Your Code Changes and Your DataThe Power and Danger of InteractivityBook Code Versus Notebook CodeDownload the DataTake a Quick Look at the Data StructureCreate a Test SetExplore and Visualize the Data to Gain InsightsVisualizing Geographical DataLook for CorrelationsExperiment with Attribute CombinationsPrepare the Data for Machine Learning AlgorithmsClean the DataHandling Text and Categorical AttributesFeature Scaling and TransformationCustom TransformersTransformation PipelinesSelect and Train a ModelTrain and Evaluate on the Training SetBetter Evaluation Using Cross-ValidationFine-Tune Your ModelGrid SearchRandomized SearchEnsemble MethodsAnalyzing the Best Models and Their ErrorsEvaluate Your System on the Test SetLaunch, Monitor, and Maintain Your SystemTry It Out!Exercises
MNISTTraining a Binary ClassifierPerformance MeasuresMeasuring Accuracy Using Cross-ValidationConfusion MatricesPrecision and RecallThe Precision/Recall Trade-offThe ROC CurveMulticlass ClassificationError AnalysisMultilabel ClassificationMultioutput ClassificationExercises
Linear RegressionThe Normal EquationComputational ComplexityGradient DescentBatch Gradient DescentStochastic Gradient DescentMini-Batch Gradient DescentPolynomial RegressionLearning CurvesRegularized Linear ModelsRidge RegressionLasso RegressionElastic Net RegressionEarly StoppingLogistic RegressionEstimating ProbabilitiesTraining and Cost FunctionDecision BoundariesSoftmax RegressionExercises
Linear SVM ClassificationSoft Margin ClassificationNonlinear SVM ClassificationPolynomial KernelSimilarity FeaturesGaussian RBF KernelSVM Classes and Computational ComplexitySVM RegressionUnder the Hood of Linear SVM ClassifiersThe Dual ProblemKernelized SVMsExercises
Training and Visualizing a Decision TreeMaking PredictionsEstimating Class ProbabilitiesThe CART Training AlgorithmComputational ComplexityGini Impurity or Entropy?Regularization HyperparametersRegressionSensitivity to Axis OrientationDecision Trees Have a High VarianceExercises
Voting ClassifiersBagging and PastingBagging and Pasting in Scikit-LearnOut-of-Bag EvaluationRandom Patches and Random SubspacesRandom ForestsExtra-TreesFeature ImportanceBoostingAdaBoostGradient BoostingHistogram-Based Gradient BoostingStackingExercises
The Curse of DimensionalityMain Approaches for Dimensionality ReductionProjectionManifold LearningPCAPreserving the VariancePrincipal ComponentsProjecting Down to d DimensionsUsing Scikit-LearnExplained Variance RatioChoosing the Right Number of DimensionsPCA for CompressionRandomized PCAIncremental PCARandom ProjectionLLEOther Dimensionality Reduction TechniquesExercises

Clustering Algorithms: k-means and DBSCANk-meansLimits of k-meansUsing Clustering for Image SegmentationUsing Clustering for Semi-Supervised LearningDBSCANOther Clustering AlgorithmsGaussian MixturesUsing Gaussian Mixtures for Anomaly DetectionSelecting the Number of ClustersBayesian Gaussian Mixture ModelsOther Algorithms for Anomaly and Novelty DetectionExercises
From Biological to Artificial NeuronsBiological NeuronsLogical Computations with NeuronsThe PerceptronThe Multilayer Perceptron and BackpropagationRegression MLPsClassification MLPsImplementing MLPs with KerasBuilding an Image Classifier Using the Sequential APIBuilding a Regression MLP Using the Sequential APIBuilding Complex Models Using the Functional APIUsing the Subclassing API to Build Dynamic ModelsSaving and Restoring a ModelUsing CallbacksUsing TensorBoard for VisualizationFine-Tuning Neural Network HyperparametersNumber of Hidden LayersNumber of Neurons per Hidden LayerLearning Rate, Batch Size, and Other HyperparametersExercises
The Vanishing/Exploding Gradients ProblemsGlorot and He InitializationBetter Activation FunctionsBatch NormalizationGradient ClippingReusing Pretrained LayersTransfer Learning with KerasUnsupervised PretrainingPretraining on an Auxiliary TaskFaster OptimizersMomentumNesterov Accelerated GradientAdaGradRMSPropAdamAdaMaxNadamAdamWLearning Rate SchedulingAvoiding Overfitting Through Regularizationℓ1 and ℓ2 RegularizationDropoutMonte Carlo (MC) DropoutMax-Norm RegularizationSummary and Practical GuidelinesExercises
A Quick Tour of TensorFlowUsing TensorFlow like NumPyTensors and OperationsTensors and NumPyType ConversionsVariablesOther Data StructuresCustomizing Models and Training AlgorithmsCustom Loss FunctionsSaving and Loading Models That Contain Custom ComponentsCustom Activation Functions, Initializers, Regularizers, and ConstraintsCustom MetricsCustom LayersCustom ModelsLosses and Metrics Based on Model InternalsComputing Gradients Using AutodiffCustom Training LoopsTensorFlow Functions and GraphsAutoGraph and TracingTF Function RulesExercises
The tf.data APIChaining TransformationsShuffling the DataInterleaving Lines from Multiple FilesPreprocessing the DataPutting Everything TogetherPrefetchingUsing the Dataset with KerasThe TFRecord FormatCompressed TFRecord FilesA Brief Introduction to Protocol BuffersTensorFlow ProtobufsLoading and Parsing ExamplesHandling Lists of Lists Using the SequenceExample ProtobufKeras Preprocessing LayersThe Normalization LayerThe Discretization LayerThe CategoryEncoding LayerThe StringLookup LayerThe Hashing LayerEncoding Categorical Features Using EmbeddingsText PreprocessingUsing Pretrained Language Model ComponentsImage Preprocessing LayersThe TensorFlow Datasets ProjectExercises
The Architecture of the Visual CortexConvolutional LayersFiltersStacking Multiple Feature MapsImplementing Convolutional Layers with KerasMemory RequirementsPooling LayersImplementing Pooling Layers with KerasCNN ArchitecturesLeNet-5AlexNetGoogLeNetVGGNetResNetXceptionSENetOther Noteworthy ArchitecturesChoosing the Right CNN ArchitectureImplementing a ResNet-34 CNN Using KerasUsing Pretrained Models from KerasPretrained Models for Transfer LearningClassification and LocalizationObject DetectionFully Convolutional NetworksYou Only Look OnceObject TrackingSemantic SegmentationExercises
Recurrent Neurons and LayersMemory CellsInput and Output SequencesTraining RNNsForecasting a Time SeriesThe ARMA Model FamilyPreparing the Data for Machine Learning ModelsForecasting Using a Linear ModelForecasting Using a Simple RNNForecasting Using a Deep RNNForecasting Multivariate Time SeriesForecasting Several Time Steps AheadForecasting Using a Sequence-to-Sequence ModelHandling Long SequencesFighting the Unstable Gradients ProblemTackling the Short-Term Memory ProblemExercises
Generating Shakespearean Text Using a Character RNNCreating the Training DatasetBuilding and Training the Char-RNN ModelGenerating Fake Shakespearean TextStateful RNNSentiment AnalysisMaskingReusing Pretrained Embeddings and Language ModelsAn Encoder–Decoder Network for Neural Machine TranslationBidirectional RNNsBeam SearchAttention MechanismsAttention Is All You Need: The Original Transformer ArchitectureAn Avalanche of Transformer ModelsVision TransformersHugging Face’s Transformers LibraryExercises
Efficient Data RepresentationsPerforming PCA with an Undercomplete Linear AutoencoderStacked AutoencodersImplementing a Stacked Autoencoder Using KerasVisualizing the ReconstructionsVisualizing the Fashion MNIST DatasetUnsupervised Pretraining Using Stacked AutoencodersTying WeightsTraining One Autoencoder at a TimeConvolutional AutoencodersDenoising AutoencodersSparse AutoencodersVariational AutoencodersGenerating Fashion MNIST ImagesGenerative Adversarial NetworksThe Difficulties of Training GANsDeep Convolutional GANsProgressive Growing of GANsStyleGANsDiffusion ModelsExercises
Learning to Optimize RewardsPolicy SearchIntroduction to OpenAI GymNeural Network PoliciesEvaluating Actions: The Credit Assignment ProblemPolicy GradientsMarkov Decision ProcessesTemporal Difference LearningQ-LearningExploration PoliciesApproximate Q-Learning and Deep Q-LearningImplementing Deep Q-LearningDeep Q-Learning VariantsFixed Q-value TargetsDouble DQNPrioritized Experience ReplayDueling DQNOverview of Some Popular RL AlgorithmsExercises
Serving a TensorFlow ModelUsing TensorFlow ServingCreating a Prediction Service on Vertex AIRunning Batch Prediction Jobs on Vertex AIDeploying a Model to a Mobile or Embedded DeviceRunning a Model in a Web PageUsing GPUs to Speed Up ComputationsGetting Your Own GPUManaging the GPU RAMPlacing Operations and Variables on DevicesParallel Execution Across Multiple DevicesTraining Models Across Multiple DevicesModel ParallelismData ParallelismTraining at Scale Using the Distribution Strategies APITraining a Model on a TensorFlow ClusterRunning Large Training Jobs on Vertex AIHyperparameter Tuning on Vertex AIExercisesThank You!
Frame the Problem and Look at the Big PictureGet the DataExplore the DataPrepare the DataShortlist Promising ModelsFine-Tune the SystemPresent Your SolutionLaunch!
Manual DifferentiationFinite Difference ApproximationForward-Mode AutodiffReverse-Mode Autodiff
StringsRagged TensorsSparse TensorsTensor ArraysSetsQueues
TF Functions and Concrete FunctionsExploring Function Definitions and GraphsA Closer Look at TracingUsing AutoGraph to Capture Control FlowHandling Variables and Other Resources in TF FunctionsUsing TF Functions with Keras (or Not)

Content preview from Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 3rd Edition

Chapter 13. Loading and Preprocessing Data with TensorFlow

In Chapter 2, you saw that loading and preprocessing data is an important part of any machine learning project. You used Pandas to load and explore the (modified) California housing dataset—which was stored in a CSV file—and you applied Scikit-Learn’s transformers for preprocessing. These tools are quite convenient, and you will probably be using them often, especially when exploring and experimenting with data.

However, when training TensorFlow models on large datasets, you may prefer to use TensorFlow’s own data loading and preprocessing API, called tf.data. It is capable of loading and preprocessing data extremely efficiently, reading from multiple files in parallel using multithreading and queuing, shuffling and batching samples, and more. Plus, it can do all of this on the fly—it loads and preprocesses the next batch of data across multiple CPU cores, while your GPUs or TPUs are busy training the current batch of data.

The tf.data API lets you handle datasets that don’t fit in memory, and it allows you to make full use of your hardware resources, thereby speeding up training. Off the shelf, the tf.data API can read from text files (such as CSV files), binary files with fixed-size records, and binary files that use TensorFlow’s TFRecord format, which supports records of varying sizes.

TFRecord is a flexible and efficient binary format usually containing protocol buffers (an open source binary format). The tf.data API ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

Publisher Resources

ISBN: 9781098125967Errata Page Supplemental Content

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 3rd Edition

by Aurélien Géron

Chapter 13. Loading and Preprocessing Data with TensorFlow

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

You might also like

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

Hands-On Machine Learning with Scikit-Learn and PyTorch

Machine Learning with PyTorch and Scikit-Learn

Deep Learning for Coders with fastai and PyTorch

Publisher Resources

Chapter 13. Loading and Preprocessing Data with TensorFlow

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

Hands-On Machine Learning with Scikit-Learn and PyTorch

Machine Learning with PyTorch and Scikit-Learn

Deep Learning for Coders with fastai and PyTorch

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.