book

Practical Deep Learning, 2nd Edition

Name: Practical Deep Learning, 2nd Edition
Author: Ronald T. Kneusel
ISBN: 9781718504202

by Ronald T. Kneusel

May 2025

Intermediate to advanced

584 pages

16h 49m

English

No Starch Press

Read now

Unlock full access

Cover Page
Title Page
Copyright Page
Dedication Page
About the Author
About the Technical Reviewer
BRIEF CONTENTS
CONTENTS IN DETAIL
FOREWORD TO THE FIRST EDITION
ACKNOWLEDGMENTS

INTRODUCTION
Who This Book Is ForWhat You Can Expect to LearnAbout This BookTerminologyWhat’s New in the Second EditionSynopsis
0 ENVIRONMENT AND MATHEMATICAL PRELIMINARIES
The Operating EnvironmentNumPyscikit-learnTensorFlow with KerasInstalling the ToolkitsBasic Linear AlgebraVectorsMatricesVector and Matrix MultiplicationStatistics and ProbabilityDescriptive StatisticsProbability DistributionsStatistical TestsGraphics Processing UnitsSummary
PART I DATA IS EVERYTHING
1 IT’S ALL ABOUT THE DATA
Classes and LabelsFeatures and Feature VectorsTypes of FeaturesFeature Selection and the Curse of DimensionalityQualities of a Good DatasetInterpolation and ExtrapolationThe Parent DistributionPrior Class ProbabilitiesConfusersDataset SizeData PreparationScaling FeaturesDealing with Missing FeaturesTraining, Validation, and Test DataThe Three SubsetsDataset Partitioningk-Fold Cross-ValidationData AnalysisHow to Find Problems in the DataCautionary TalesSummary
2 BUILDING THE DATASETS
IrisesBreast CancerMNIST DigitsCIFAR-10Data AugmentationReasoningMethodsThe Iris DatasetThe CIFAR-10 DatasetSummary
PART II CLASSICAL MACHINE LEARNING
3 INTRODUCTION TO MACHINE LEARNING
Nearest Centroidk-Nearest NeighborsNaive BayesTree ClassifiersRecursion PrimerDecision TreesRandom ForestsSupport Vector MachinesMarginsSupport VectorsOptimizationKernelsSummary
4 EXPERIMENTS WITH CLASSICAL MODELS
Experiments with the Iris DatasetTesting the Classical ModelsImplementing a Nearest-Centroid ClassifierExperiments with the Breast Cancer DatasetRunning Two Initial TestsTesting the Effect of Random SplitsAdding k-Fold ValidationSearching for HyperparametersExperiments with the MNIST DatasetTesting the Classical ModelsAnalyzing RuntimesExperimenting with PCA ComponentsScrambling the DatasetClassical Model SummaryNearest Centroidk-Nearest NeighborsNaive BayesDecision TreesRandom ForestsSupport Vector MachinesWhen to Use Classical ModelsHandling Small DatasetsDealing with Reduced Computational RequirementsHaving Explainable ModelsWorking with Vector InputsSummary
PART III NEURAL NETWORKS
5 INTRODUCTION TO NEURAL NETWORKS
Anatomy of a Neural NetworkThe NeuronActivation FunctionsThe Architecture of a NetworkOutput LayersWeight and Bias RepresentationA Simple Neural Network ImplementationBuilding the DatasetImplementing the Neural NetworkTraining and Testing the Neural NetworkSummary
6 TRAINING A NEURAL NETWORK
A High-Level OverviewGradient DescentFinding MinimumsUpdating the WeightsStochastic Gradient DescentBatches and MinibatchesConvex vs. Nonconvex FunctionsWhen to Stop TrainingThe Learning RateMomentumBackpropagationA Simple ExampleAn Abstract ExampleLoss FunctionsAbsolute and Mean Squared Error LossCross-Entropy LossWeight InitializationManaging Model Complexity and GeneralizationOverfittingRegularizationL2 RegularizationDropoutSummary
7 EXPERIMENTS WITH NEURAL NETWORKS
The DatasetThe MLPClassifier ClassArchitecture and Activation FunctionsThe CodeThe ResultsBatch SizeBase Learning RateTraining-Set SizeL2 RegularizationMomentumWeight InitializationFeature OrderingSummary
8 EVALUATING MODELS
Definitions and AssumptionsWhy Accuracy Is Not EnoughThe 2×2 Confusion MatrixMetrics Derived from the 2×2 Confusion MatrixDeriving Metrics from the 2×2 TableUsing Metrics to Interpret ModelsMore-Advanced MetricsInformedness and MarkednessF1 ScoreCohen’s KappaMatthews Correlation CoefficientMetric ImplementationThe Receiver Operating Characteristics CurveGathering the ModelsPlotting the MetricsExploring the ROC CurveComparing Models with ROC AnalysisGenerating an ROC CurveHandling Multiple ClassesExtending the Confusion MatrixCalculating Weighted AccuracyConsidering the Multiclass Matthews Correlation CoefficientSummary
PART IV CONVOLUTIONAL NEURAL NETWORKS
9 INTRODUCTION TO CONVOLUTIONAL NEURAL NETWORKS
Why Convolutional Neural Networks?ConvolutionScanning with the KernelUsing Convolution for Image ProcessingAnatomy of a Convolutional Neural NetworkExploring the Types of LayersPassing Data Through the CNNConvolutional LayersHow They WorkIn ActionMultiple LayersInitializationPooling LayersFully Connected LayersFully Convolutional LayersHow the CNN OperatesSummary
10 EXPERIMENTS WITH KERAS AND MNIST
Building CNNs in KerasLoading the MNIST DataBuilding the ModelTraining and Evaluating the ModelPlotting the ErrorBasic ExperimentsArchitecture ExperimentsTraining-Set Size, Minibatches, and EpochsOptimizersFully Convolutional NetworksBuilding and Training the ModelMaking the Test ImagesTesting the ModelScrambled MNIST DigitsSummary
11 EXPERIMENTS WITH CIFAR-10
A CIFAR-10 RefresherThe Full CIFAR-10 DatasetBuilding the ModelsAnalyzing the ModelsAnimal or Vehicle?Binary or Multiclass?Using the Augmented CIFAR-10 DatasetSummary
12 A CASE STUDY: CLASSIFYING AUDIO SAMPLES
Building the DatasetAugmenting the DatasetPreprocessing the DataClassifying the Audio FeaturesWith Classical ModelsWith a Traditional Neural NetworkWith a Convolutional Neural NetworkSpectrogramsClassifying SpectrogramsEnsemblesSummary
PART V ADVANCED NETWORKS AND GENERATIVE AI
13 ADVANCED CNN ARCHITECTURES
The Keras Functional APIVGGStandardizing with Batch NormalizationApplying Dropout After Convolutional LayersBuilding the VGG8 ModelTesting the VGG8 ModelResNetMitigating the Vanishing Gradient ProblemExploring ResNet ConfigurationsBuilding the ResNet-18 ModelTesting the ResNet-18 ModelMobileNetImplementing Depthwise ConvolutionsBuilding the MobileNet ModelTesting the MobileNet ModelBuilding an EnsembleSummary
14 FINE-TUNING AND TRANSFER LEARNING
Fine-Tuning a Pretrained ModelVGG16 and MobileNet with CIFAR-10ExperimentsA Study in Fine-TuningCreating New Features with Transfer LearningExtracting CIFAR-10 Features with VGG16Training Classical Models with CIFAR-10 FeaturesDetecting Anomalies with CIFAR-10 FeaturesRetrieving Images with CIFAR-10 FeaturesSummary
15 FROM CLASSIFICATION TO LOCALIZATION
Detection Experiments with MNISTBuilding the DatasetBuilding the ModelRunning the ModelRunning an Advanced Detection ModelSemantic Segmentation with U-NetImplementing the U-Net ModelRunning the Model and Interpreting Its OutputMultilabel ClassificationDatasetPerformanceSummary
16 SELF-SUPERVISED LEARNING
Building the Unlabeled DatasetRotation PredictionBuilding and Testing RotNetExploring Variations on the RotNet ThemeExamining RotNet FeaturesFine-Tuning ExperimentsSiamese NetworksBuilding and Testing the Siamese NetworksExamining Siamese Network FeaturesFine-Tuning ExperimentsFor Further ExplorationSummary
17 GENERATIVE ADVERSARIAL NETWORKS
How GANs WorkUnconditional GANsBuilding a GAN with Multilayer PerceptronsBuilding a GAN with Convolutional LayersExperimenting with Unconditional GANsConditional GANsImplementing a Conditional GANExperimenting with Conditional GANsExploring the Latent SpaceIntrinsic DimensionalityLatent Space InterpolationSummary
18 LARGE LANGUAGE MODELS
Understanding Large Language ModelsEvaluating the LLM Block DiagramTokenizing and EmbeddingExploring the Transformer LayerPredicting the Next TokenIn-Context LearningConfiguring the ExperimentRunning the ExperimentTesting Another LLMRunning LLMs LocallyRepresenting Parameters with QuantizationInstalling OllamaInstalling Open Source LLMsChatting with a Simple ChatbotLLM EmbeddingsEmbeddings Encode MeaningSemantic Search and Retrieval-Augmented GenerationSentiment AnalysisLLMs and ImagesDescribing ImagesClassifying Images with an LLM?Are LLMs Creative?The Divergent Association TaskDAT Scores as a Function of TemperatureCreative Writing as a Function of TemperatureSummary
AFTERWORD
INDEX

Content preview from Practical Deep Learning, 2nd Edition

5INTRODUCTION TO NEURAL NETWORKS

A snapshot from the book’s cover that functions as a chapter opening image. Shows a robot attempting to lift a large boulder with an integral symbol.

Neural networks are the heart of deep learning. Before we dive into what we’ll later call traditional neural networks in Chapter 6, we’ll spend this chapter introducing the anatomy of a neural network, followed by a quick example.

Specifically, we’ll present the components of a fully connected feedforward neural network. You can visually imagine the network as shown in Figure 5-1. We’ll refer to this figure often throughout this chapter and the next. Your mission, should you choose to accept it, is to commit this figure to memory to save wear and tear on the book by flipping back to it repeatedly.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9798341655119Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Practical Deep Learning, 2nd Edition

by Ronald T. Kneusel

5INTRODUCTION TO NEURAL NETWORKS

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.