book

Learning Deep Learning: Theory and Practice of Neural Networks, Computer Vision, NLP, and Transformers using TensorFlow

by Magnus Ekman

August 2021

Intermediate to advanced

752 pages

22h 5m

English

Addison-Wesley Professional

Read now

Unlock full access

Includes

Has Sandbox

What Is Deep Learning?Brief History of Deep Neural NetworksIs This Book for You?Is DL Dangerous?Choosing a DL FrameworkPrerequisites for Learning DLAbout the Code ExamplesHow to Read This BookOverview of Each Chapter and Appendix

Example of a Two-Input PerceptronThe Perceptron Learning AlgorithmLimitations of the PerceptronCombining Multiple PerceptronsImplementing Perceptrons with Linear AlgebraGeometric Interpretation of the PerceptronUnderstanding the Bias TermConcluding Remarks on the Perceptron
Intuitive Explanation of the Perceptron Learning AlgorithmDerivatives and Optimization ProblemsSolving a Learning Problem with Gradient DescentConstants and Variables in a NetworkAnalytic Explanation of the Perceptron Learning AlgorithmGeometric Description of the Perceptron Learning AlgorithmRevisiting Different Types of Perceptron PlotsUsing a Perceptron to Identify PatternsConcluding Remarks on Gradient-Based Learning
Modified Neurons to Enable Gradient Descent for Multilevel NetworksWhich Activation Function Should We Use?Function Composition and the Chain RuleUsing Backpropagation to Compute the GradientBackpropagation with Multiple Neurons per LayerProgramming Example: Learning the XOR FunctionNetwork ArchitecturesConcluding Remarks on Backpropagation
Introduction to Datasets Used When Training NetworksTraining and InferenceExtending the Network and Learning Algorithm to Do Multiclass ClassificationNetwork for Digit ClassificationLoss Function for Multiclass ClassificationProgramming Example: Classifying Handwritten DigitsMini-Batch Gradient DescentConcluding Remarks on Multiclass Classification
Programming Example: Moving to a DL FrameworkThe Problem of Saturated Neurons and Vanishing GradientsInitialization and Normalization Techniques to Avoid Saturated NeuronsCross-Entropy Loss Function to Mitigate Effect of Saturated Output NeuronsDifferent Activation Functions to Avoid Vanishing Gradient in Hidden LayersVariations on Gradient Descent to Improve LearningExperiment: Tweaking Network and Learning ParametersHyperparameter Tuning and Cross-ValidationConcluding Remarks on the Path Toward Deep Learning
Output UnitsThe Boston Housing DatasetProgramming Example: Predicting House Prices with a DNNImproving Generalization with RegularizationExperiment: Deeper and Regularized Models for House Price PredictionConcluding Remarks on Output Units and Regression Problems
The CIFAR-10 DatasetCharacteristics and Building Blocks for Convolutional LayersCombining Feature Maps into a Convolutional LayerCombining Convolutional and Fully Connected Layers into a NetworkEffects of Sparse Connections and Weight SharingProgramming Example: Image Classification with a Convolutional NetworkConcluding Remarks on Convolutional Networks
VGGNetGoogLeNetResNetProgramming Example: Use a Pretrained ResNet ImplementationTransfer LearningBackpropagation for CNN and PoolingData Augmentation as a Regularization TechniqueMistakes Made by CNNsReducing Parameters with Depthwise Separable ConvolutionsStriking the Right Network Design Balance with EfficientNetConcluding Remarks on Deeper CNNs
Limitations of Feedforward NetworksRecurrent Neural NetworksMathematical Representation of a Recurrent LayerCombining Layers into an RNNAlternative View of RNN and Unrolling in TimeBackpropagation Through TimeProgramming Example: Forecasting Book SalesDataset Considerations for RNNsConcluding Remarks on RNNs
Keeping Gradients HealthyIntroduction to LSTMAlternative View of LSTMRelated Topics: Highway Networks and Skip ConnectionsConcluding Remarks on LSTM
Encoding TextLonger-Term Prediction and Autoregressive ModelsBeam SearchProgramming Example: Using LSTM for Text AutocompletionBidirectional RNNsDifferent Combinations of Input and Output SequencesConcluding Remarks on Text Autocompletion with LSTM
Introduction to Language Models and Their Use CasesExamples of Different Language ModelsBenefit of Word Embeddings and Insight into How They WorkWord Embeddings Created by Neural Language ModelsProgramming Example: Neural Language Model and Resulting EmbeddingsKing – Man + Woman = QueenKing – Man + Woman ! = QueenLanguage Models, Word Embeddings, and Human BiasesRelated Topic: Sentiment Analysis of TextConcluding Remarks on Language Models and Word Embeddings
Using word2vec to Create Word Embeddings Without a Language ModelAdditional Thoughts on word2vecword2vec in Matrix FormWrapping Up word2vecProgramming Example: Exploring Properties of GloVe EmbeddingsConcluding Remarks on word2vec and GloVe
Encoder-Decoder Model for Sequence-to-Sequence LearningIntroduction to the Keras Functional APIProgramming Example: Neural Machine TranslationExperimental ResultsProperties of the Intermediate RepresentationConcluding Remarks on Language Translation
Rationale Behind AttentionAttention in Sequence-to-Sequence NetworksAlternatives to Recurrent NetworksSelf-AttentionMulti-head AttentionThe TransformerConcluding Remarks on the Transformer
Extending the Image Captioning Network with AttentionProgramming Example: Attention-Based Image CaptioningConcluding Remarks on Image Captioning
AutoencodersMultimodal LearningMultitask LearningProcess for Tuning a NetworkNeural Architecture SearchConcluding Remarks
Things You Should Know by NowEthical AI and Data EthicsThings You Do Not Yet KnowNext Steps
Linear Regression as a Machine Learning AlgorithmComputing Linear Regression CoefficientsClassification with Logistic RegressionClassifying XOR with a Linear ClassifierClassification with Support Vector MachinesEvaluation Metrics for a Binary Classifier
Object DetectionSemantic SegmentationInstance Segmentation with Mask R-CNN
WordpiecesFastTextCharacter-Based MethodELMoRelated Work
GPTBERTRoBERTaHistorical Work Leading Up to GPT and BERTOther Models Based on the Transformer
Newton-Raphson Root-Finding MethodRelationship Between Newton-Raphson and Gradient Descent
Single MatrixMini-Batch Implementation
Alternative GRU ImplementationNetwork Based on the GRU
PythonProgramming EnvironmentProgramming ExamplesDatasetsInstalling a DL FrameworkTensorFlow Specific ConsiderationsKey Differences Between PyTorch and TensorFlow

Content preview from Learning Deep Learning: Theory and Practice of Neural Networks, Computer Vision, NLP, and Transformers using TensorFlow

About the Author

Magnus Ekman, PhD, is a Director of Architecture at NVIDIA Corporation. His doctorate is in computer engineering, and he holds multiple patents. He was first exposed to artificial neural networks in the late 1990s in his native country, Sweden. After some dabbling in evolutionary computation, he focused on computer architecture and relocated to Silicon Valley, where he lives with his wife, Jennifer, children, Sebastian and Sofia, and dog, Babette. He has previously worked with processor design and R&D at Sun Microsystems and Samsung Research America and has been involved in starting two companies, one of which (Skout) was later acquired by The Meet Group, Inc. In his current role at NVIDIA, he leads an engineering team working ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

Publisher Resources

ISBN: 9780137470198

Learning Deep Learning: Theory and Practice of Neural Networks, Computer Vision, NLP, and Transformers using TensorFlow

by Magnus Ekman

About the Author

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

You might also like

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

Deep Learning with TensorFlow, Keras, and PyTorch

Generative Deep Learning, 2nd Edition

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 3rd Edition

Publisher Resources

About the Author

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

Deep Learning with TensorFlow, Keras, and PyTorch

Generative Deep Learning, 2nd Edition

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 3rd Edition

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.