book

Deep Learning from the Basics

Name: Deep Learning from the Basics
ISBN: 9781800206137

by Koki Saitoh, Shigeo Yushita

March 2021

Beginner to intermediate

316 pages

6h 34m

English

Packt Publishing

Read now

Unlock full access

Preface
About the BookAbout the AuthorsLearning ObjectivesAudienceApproach
Introduction
Concept of this bookFor whom is this book written?For whom is this book not written?How to read this bookAcknowledgments
1. Introduction to Python
What is Python?Installing PythonPython VersionsExternal Libraries That We UseAnaconda DistributionPython InterpreterMathematical OperationsData TypesVariablesListsDictionariesBooleanif Statementsfor StatementsFunctionsPython Script FilesSaving in a FileClassesNumPyImporting NumPyCreating a NumPy ArrayMathematical Operations in NumPyN-Dimensional NumPy ArraysBroadcastingAccessing ElementsMatplotlibDrawing a Simple GraphFeatures of pyplotDisplaying ImagesSummary
2. Perceptrons
What Is a Perceptron?Simple Logic CircuitsAND GateNAND and OR gatesImplementing PerceptronsEasy ImplementationIntroducing Weights and BiasImplementation with Weights and BiasLimitations of PerceptronsXOR GateLinear and NonlinearMultilayer PerceptronsCombining the Existing GatesImplementing an XOR GateFrom NAND to a ComputerSummary
3. Neural Networks
From Perceptrons to Neural NetworksNeural Network ExampleReviewing the PerceptronIntroducing an Activation FunctionActivation FunctionSigmoid FunctionImplementing a Step FunctionStep Function GraphImplementing a Sigmoid FunctionComparing the Sigmoid Function and the Step FunctionNonlinear FunctionReLU FunctionCalculating Multidimensional ArraysMultidimensional ArraysMatrix MultiplicationMatrix Multiplication in a Neural NetworkImplementing a Three-Layer Neural NetworkExamining the SymbolsImplementing Signal Transmission in Each LayerImplementation SummaryDesigning the Output LayerIdentity Function and Softmax FunctionIssues when Implementing the Softmax FunctionCharacteristics of the Softmax FunctionNumber of Neurons in the Output LayerHandwritten Digit RecognitionMNIST DatasetInference for Neural NetworkBatch ProcessingSummary
4. Neural Network Training
Learning from DataData-DrivenTraining Data and Test DataLoss FunctionSum of Squared ErrorsCross-Entropy ErrorMini-Batch LearningImplementing Cross-Entropy Error (Using Batches)Why Do We Configure a Loss Function?Numerical DifferentiationDerivativeExamples of Numerical DifferentiationPartial DerivativeGradientGradient MethodGradients for a Neural NetworkImplementing a Training AlgorithmA Two-Layer Neural Network as a ClassImplementing Mini-Batch TrainingUsing Test Data for EvaluationSummary
5. Backpropagation
Computational GraphsUsing Computational Graphs to Solve ProblemsLocal CalculationWhy Do We Use Computational Graphs?Chain RuleBackward Propagation in a Computational GraphWhat Is the Chain Rule?The Chain Rule and Computational GraphsBackward PropagationBackward Propagation in an Addition NodeBackward Propagation in a Multiplication NodeApples ExampleImplementing a Simple LayerImplementing a Multiplication LayerImplementing an Addition LayerImplementing the Activation Function LayerReLU LayerSigmoid LayerImplementing the Affine and Softmax LayersAffine LayerBatch-Based Affine LayerSoftmax-with-Loss LayerImplementing BackpropagationOverall View of Neural Network TrainingPresuppositionImplementing a Neural Network That Supports BackpropagationGradient CheckTraining Using BackpropagationSummary
6. Training Techniques
Updating ParametersStory of an AdventurerSGDDisadvantage of SGDMomentumAdaGradAdamWhich Update Technique Should We Use?Using the MNIST Dataset to Compare the Update TechniquesInitial Weight ValuesHow About Setting the Initial Weight Values to 0?Distribution of Activations in the Hidden LayersInitial Weight Values for ReLUUsing the MNIST Dataset to Compare the Weight InitializersBatch NormalizationBatch Normalization AlgorithmEvaluating Batch NormalizationRegularizationOverfittingWeight DecayDropoutValidating HyperparametersValidation DataOptimizing HyperparametersImplementing Hyperparameter OptimizationSummary
7. Convolutional Neural Networks
Overall ArchitectureThe Convolution LayerIssues with the Fully Connected LayerConvolution OperationsPaddingStridePerforming a Convolution Operation on Three-Dimensional DataThinking in BlocksBatch ProcessingThe Pooling LayerCharacteristics of a Pooling LayerImplementing the Convolution and Pooling LayersFour-Dimensional ArraysExpansion by im2colImplementing a Convolution LayerImplementing a Pooling LayerImplementing a CNNVisualizing a CNNVisualizing the Weight of the First LayerUsing a Hierarchical Structure to Extract InformationTypical CNNsLeNetAlexNetSummary
8. Deep Learning
Making a Network DeeperDeeper NetworksImproving Recognition AccuracyMotivation for a Deeper NetworkA Brief History of Deep LearningImageNetVGGGoogLeNetResNetAccelerating Deep LearningChallenges to OvercomeUsing GPUs for AccelerationDistributed TrainingReducing the Bit Number for Arithmetic PrecisionPractical Uses of Deep LearningObject DetectionSegmentationGenerating Image CaptionsThe Future of Deep LearningConverting Image StylesGenerating ImagesAutomated DrivingDeep Q-Networks (Reinforcement Learning)Summary

Appendix A
Computational Graph of the Softmax-with-Loss LayerForward PropagationBackward PropagationSummary

Content preview from Deep Learning from the Basics

Preface

About

This section briefly introduces the author and the material covered in this book.

About the Book

Deep learning is rapidly becoming the most preferred way of solving data problems. This is thanks, in part, to its huge variety of mathematical algorithms and their ability to find patterns that are otherwise invisible to us.

Deep Learning from the Basics begins with a fast-paced introduction to deep learning with Python, its definition, characteristics, and applications. You'll learn how to use the Python interpreter for the script files. You'll also learn how to utilize NumPy and Matplotlib in your deep learning applications. As you progress through the book, you'll discover backpropagation—an efficient way to calculate the ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781800206137

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design