book

Learning Deep Learning: Theory and Practice of Neural Networks, Computer Vision, NLP, and Transformers using TensorFlow

by Magnus Ekman

August 2021

Intermediate to advanced

752 pages

22h 5m

English

Addison-Wesley Professional

Read now

Unlock full access

Includes

Has Sandbox

What Is Deep Learning?Brief History of Deep Neural NetworksIs This Book for You?Is DL Dangerous?Choosing a DL FrameworkPrerequisites for Learning DLAbout the Code ExamplesHow to Read This BookOverview of Each Chapter and Appendix

Example of a Two-Input PerceptronThe Perceptron Learning AlgorithmLimitations of the PerceptronCombining Multiple PerceptronsImplementing Perceptrons with Linear AlgebraGeometric Interpretation of the PerceptronUnderstanding the Bias TermConcluding Remarks on the Perceptron
Intuitive Explanation of the Perceptron Learning AlgorithmDerivatives and Optimization ProblemsSolving a Learning Problem with Gradient DescentConstants and Variables in a NetworkAnalytic Explanation of the Perceptron Learning AlgorithmGeometric Description of the Perceptron Learning AlgorithmRevisiting Different Types of Perceptron PlotsUsing a Perceptron to Identify PatternsConcluding Remarks on Gradient-Based Learning
Modified Neurons to Enable Gradient Descent for Multilevel NetworksWhich Activation Function Should We Use?Function Composition and the Chain RuleUsing Backpropagation to Compute the GradientBackpropagation with Multiple Neurons per LayerProgramming Example: Learning the XOR FunctionNetwork ArchitecturesConcluding Remarks on Backpropagation
Introduction to Datasets Used When Training NetworksTraining and InferenceExtending the Network and Learning Algorithm to Do Multiclass ClassificationNetwork for Digit ClassificationLoss Function for Multiclass ClassificationProgramming Example: Classifying Handwritten DigitsMini-Batch Gradient DescentConcluding Remarks on Multiclass Classification
Programming Example: Moving to a DL FrameworkThe Problem of Saturated Neurons and Vanishing GradientsInitialization and Normalization Techniques to Avoid Saturated NeuronsCross-Entropy Loss Function to Mitigate Effect of Saturated Output NeuronsDifferent Activation Functions to Avoid Vanishing Gradient in Hidden LayersVariations on Gradient Descent to Improve LearningExperiment: Tweaking Network and Learning ParametersHyperparameter Tuning and Cross-ValidationConcluding Remarks on the Path Toward Deep Learning
Output UnitsThe Boston Housing DatasetProgramming Example: Predicting House Prices with a DNNImproving Generalization with RegularizationExperiment: Deeper and Regularized Models for House Price PredictionConcluding Remarks on Output Units and Regression Problems
The CIFAR-10 DatasetCharacteristics and Building Blocks for Convolutional LayersCombining Feature Maps into a Convolutional LayerCombining Convolutional and Fully Connected Layers into a NetworkEffects of Sparse Connections and Weight SharingProgramming Example: Image Classification with a Convolutional NetworkConcluding Remarks on Convolutional Networks
VGGNetGoogLeNetResNetProgramming Example: Use a Pretrained ResNet ImplementationTransfer LearningBackpropagation for CNN and PoolingData Augmentation as a Regularization TechniqueMistakes Made by CNNsReducing Parameters with Depthwise Separable ConvolutionsStriking the Right Network Design Balance with EfficientNetConcluding Remarks on Deeper CNNs
Limitations of Feedforward NetworksRecurrent Neural NetworksMathematical Representation of a Recurrent LayerCombining Layers into an RNNAlternative View of RNN and Unrolling in TimeBackpropagation Through TimeProgramming Example: Forecasting Book SalesDataset Considerations for RNNsConcluding Remarks on RNNs
Keeping Gradients HealthyIntroduction to LSTMAlternative View of LSTMRelated Topics: Highway Networks and Skip ConnectionsConcluding Remarks on LSTM
Encoding TextLonger-Term Prediction and Autoregressive ModelsBeam SearchProgramming Example: Using LSTM for Text AutocompletionBidirectional RNNsDifferent Combinations of Input and Output SequencesConcluding Remarks on Text Autocompletion with LSTM
Introduction to Language Models and Their Use CasesExamples of Different Language ModelsBenefit of Word Embeddings and Insight into How They WorkWord Embeddings Created by Neural Language ModelsProgramming Example: Neural Language Model and Resulting EmbeddingsKing – Man + Woman = QueenKing – Man + Woman ! = QueenLanguage Models, Word Embeddings, and Human BiasesRelated Topic: Sentiment Analysis of TextConcluding Remarks on Language Models and Word Embeddings
Using word2vec to Create Word Embeddings Without a Language ModelAdditional Thoughts on word2vecword2vec in Matrix FormWrapping Up word2vecProgramming Example: Exploring Properties of GloVe EmbeddingsConcluding Remarks on word2vec and GloVe
Encoder-Decoder Model for Sequence-to-Sequence LearningIntroduction to the Keras Functional APIProgramming Example: Neural Machine TranslationExperimental ResultsProperties of the Intermediate RepresentationConcluding Remarks on Language Translation
Rationale Behind AttentionAttention in Sequence-to-Sequence NetworksAlternatives to Recurrent NetworksSelf-AttentionMulti-head AttentionThe TransformerConcluding Remarks on the Transformer
Extending the Image Captioning Network with AttentionProgramming Example: Attention-Based Image CaptioningConcluding Remarks on Image Captioning
AutoencodersMultimodal LearningMultitask LearningProcess for Tuning a NetworkNeural Architecture SearchConcluding Remarks
Things You Should Know by NowEthical AI and Data EthicsThings You Do Not Yet KnowNext Steps
Linear Regression as a Machine Learning AlgorithmComputing Linear Regression CoefficientsClassification with Logistic RegressionClassifying XOR with a Linear ClassifierClassification with Support Vector MachinesEvaluation Metrics for a Binary Classifier
Object DetectionSemantic SegmentationInstance Segmentation with Mask R-CNN
WordpiecesFastTextCharacter-Based MethodELMoRelated Work
GPTBERTRoBERTaHistorical Work Leading Up to GPT and BERTOther Models Based on the Transformer
Newton-Raphson Root-Finding MethodRelationship Between Newton-Raphson and Gradient Descent
Single MatrixMini-Batch Implementation
Alternative GRU ImplementationNetwork Based on the GRU
PythonProgramming EnvironmentProgramming ExamplesDatasetsInstalling a DL FrameworkTensorFlow Specific ConsiderationsKey Differences Between PyTorch and TensorFlow