book

Deep Learning

Name: Deep Learning
Author: S.Sridhar
ISBN: 9789367138663

by S.Sridhar

February 2025

Intermediate to advanced

696 pages

15h 30m

English

Pearson India

Read now

Unlock full access

Cover
About Pearson
Title Page
Table of Contents
Preface
About the Authors
Acknowledgments
List of Reviewers
1. Introduction to Deep Learning
1.1 Overview of Deep Learning1.2 Introduction to Artificial Intelligence1.2.1 Classical Period1.2.2 Symbolism Period1.3 Machine Learning Explained 1.3.1 Types of Machine Learning1.4 Artificial Neural Networks and Deep Learning Explained 1.4.1 How does Deep Learning Work?1.5 Challenges of Deep Learning1.6 History of Deep Learning1.6.1 Early Period1.6.2 Connectionism1.6.3 Deep Neural Networks1.7 Deep Learning Applications1.7.1 Natural Language Processing 1.7.2 Generative AI 1.7.3 Image Processing and Vision1.7.4 Navigation Systems1.7.5 Business and marketing1.7.6 IoT Applications1.7.7 Game design1.7.8 Web Analysis1.7.9 Security and Surveillance
2. Introduction to Artificial Neural Networks
2.1 What is a Neuron? 2.1.1 McCulloch-Pitts (MP) Model of an Artificial Neuron 2.1.2 What is the Use of the McCulloch-Pitts Model? 2.2 Perceptron Model 2.2.1 What are the Advantages and Disadvantages of Perceptron? 2.2.2 Perceptron Learning Algorithm 2.3 Perceptron as a Binary Classifier2.3.1 How does Classification Work in Perceptron?2.3.2 Algorithm for Using Perceptron as a Binary Classifier2.4 Single and Multilayer Neural Networks 2.4.1 Single-layer Neural Networks 2.4.2 Multilayer Networks

3. Introduction to Activation and Loss Functions
3.1 Overview of Shallow Networks 3.1.1 Shallow vs. Deep Neural Networks3.2 The Motivation of Activation Functions3.3 Types of Activation Functions3.3.1 Identity Activation Function3.3.2 Binary Step Function3.3.3 Sigmoid Activation Functions3.3.4 Hyperbolic Tangent function (Tanh Function)3.3.5 ReLU Function3.3.6 Swish Function3.3.7 Softmax Activation Function3.3.8 Softplus 3.4 Loss Functions3.4.1 Regression Loss functions3.4.2 Classification Based Error Functions3.5 Mathematics of Backpropagation 3.5.1 Role of Computational Graph and Chain rule3.5.2 Backpropagation Algorithm
4. Introduction to Optimization
4.1 Overview of Optimization4.1.1 Optimization Problems4.1.2 Optimization Algorithms4.2 Search Space 4.3 Types of Optimization Problems4.4 Hyperparameter Tuning 4.4.1 Random Search (Random Optimization or Random Sampling) 4.4.2 Grid Search4.5 Gradient Descent Algorithms4.5.1 Batch Gradient Descent 4.5.2 Stochastic Gradient Descent (SGD)4.5.3 Mini Batch4.5.4 Concept of Momentum4.5.5 Traditional and Nesterov Momentum4.5.6 Gradient Descent with Momentum4.5.7 AdaGrad (Adaptive Gradient Descent)4.5.8 RMSProp4.5.9 ADAM4.6 Second-Order Optimization Algorithms4.6.1 Newton’s Method4.6.2 Quasi-Newton Method
5. Introduction to Regularization
5.1 Introduction to Generalization and Regularization5.2 Overview of Overfitting and Underfitting5.2.1 Underfitting5.2.2 Overfitting5.3 Introduction to Regularization 5.3.1 Weight Initialization5.3.2 Network Configuration5.4 Regularization Techniques5.4.1 L1 Regularization5.4.2 L2 Regularization5.4.3 Elastic Net5.5 Neural Network Regularization5.5.1 Early Stopping5.5.2 Dropout 5.5.3 Residual Network5.5.4 Batch and Layer Normalization 5.5.5 Data Augumentation
6. Understanding Data
6.1 What is Data?6.2 Types of Data6.2.1 Structured Data6.2.2 Unstructured Data6.2.3 Semi-structured Data6.3 Data Variables and Scales6.4 Data Encoding6.5 Data Management 6.6 Data Collection 6.7 Preprocessing6.7.1 Missing Data Analysis 6.7.2 Removal of Noisy or Outlier Data6.7.3 Data Scaling6.8 Dataset Preparation 6.8.1 Resampling 6.9 Model Creation and Evaluation
7. Introduction to Regression and Classification
7.1 Introduction to Regression7.2 Linear Regression Techniques 7.2.1 Ordinary Least Square (OLS) Method 7.2.2 Matrix Methods for Linear Regression7.2.3 Linear Regression Using Stochastic Gradient Method7.2.4 Neural Networks for Linear Regression7.2.5 Regression Model Evaluation7.3 Logistic Regression7.3.1 Neural Networks for Logistic Regression7.4 Introduction to Classification7.4.1 Binary Classification7.4.2 Multiclass Classification7.5 Classifier Model Evaluation 7.5.1 Confusion Matrix7.6 Multiple Classifier Comparisons 7.6.1 ROC Curve7.6.2 Precision-Recall Curve
8. Introduction to Computer Vision and Image Processing
8.1 Introduction to Computer Vision8.1.1 Image Processing Pipeline8.2 Basics of Image Processing8.2.1 Types of Images8.3 Image Operations8.3.1 Image Arithmetic Operations8.3.2 Logical Operations8.3.3 Geometrical Operations8.3.4 Additional Operations for Data Augmentation8.4 Downsampling and Upsampling8.4.1 Image Downsampling8.4.2 Image Upsampling8.5 Convolution Operations8.5.1 Variants of Convolution Operation8.5.2 Convolution and Preprocessing8.5.3 Convolution and Feature Extraction8.6 Image Classification using Multilayer Neural Network
9. Convolutional Neural Networks
9.1 Introduction to Convolutional Neural Networks (CNN) 9.2 Machine Learning Approach Vs. Deep Learning Approach9.3 Introduction to Convolutional and Hierarchical Learning9.3.1 Hierarchical Learning9.4 Convolutional Neural Network Vs. Shallow Networks9.4.1 Differences in Architecture9.4.2 Translational Invariance9.4.3 Sparse Connectivity9.4.4 Complex Tasks9.4.5 Weight Sharing9.5 Generic CNN Architecture9.5.1 Input Layer9.5.2 Convolutional Layer9.6 Parameters and Hyperparameters of CNN9.6.1 Number of Filters and Kernel Size9.6.2 Stride9.6.3 Padding9.6.4 Pooling9.6.5 Additional Layers of CNN9.6.6 Fully Connected Layers and Image Flattening9.7 LeNet-5 Architecture
10. Transfer Learning
10.1 Overview of Transfer Learning10.1.1 Context of Transfer Learning10.2 Framework of Transfer Learning10.2.1 Components of Transfer Learning and Types of Transfer Learning Models10.3 ImageNet and Pretrained Models10.4 Advanced CNN Architectures10.4.1 AlexNet Architecture10.4.2 VGGNet 10.4.3 Inception10.4.4 ResNet10.5 Transfer Learning Models10.5.1 Using a Classifier10.5.2 Using as a Feature Extractor10.5.3 Fine-tuning10.5.4 Level of Fine Tuning
11. Introduction to Object Detection
11.1 Introduction to Object Detection11.2 Types of Object Detection11.3 Object Detection Framework11.4 Region-Based Convolutional Neural Networks (R-CNN) Family of Algorithms 11.4.1 Region-Based Convolutional Neural Networks (R-CNN) 11.4.2 Fast R-CNN11.4.3 Faster R-CNN11.5 One-Shot Detectors11.5.1 Single-Shot Multibox Detector (SSD)11.5.2 YOLO Family of Algorithms11.6 Image Segmentation11.6.1 Fully Convolutional Network (FCN) for Semantic Segmentation11.6.2 U-Net11.6.3 Mask R-CNN
12. Recurrent Neural Networks
12.1 Overview of Sequence Data12.2 Need for RNN12.2.1 Memory and Context12.3 Topology of RNN12.4 Backpropagation in Time Algorithm12.4.1 Forward Propagation12.4.2 Backpropagation12.5 Long Short-Term Memory (LSTM)12.5.1 LSTM Cell12.5.2 LSTM Gates12.6 Various Models of LSTM12.6.1 Vanilla LSTM12.6.2 Stacked LSTM12.6.3 CNN LSTM12.6.4 Encoder-Decoder LSTM12.6.5 Bi-directional LSTM12.7 Gated Recurrent Units (GRU)12.7.1 GRU Gates12.8 Case Studies for LSTM12.8.1 Case Study 1: Sentiment Analysis12.8.2 Case Study 2: Language Translation using Encoder-Decoder LSTM
13. Introduction To Autoencoders
13.1 Overview of Autoencoders 13.1.1 Encoder13.1.2 Latent Space (or Code)13.1.3 Decoder13.2 Applications of Autoencoders13.3 Implementation of an Autoencoder Using Neural Networks13.3.1 Undercomplete and Overcomplete Autoencoders13.4 Autoencoder Variants13.4.1 Vanilla Autoencoder13.4.2 Multilayer Autoencoder13.4.3 Stacked Autoencoder13.4.4 Convolutional Autoencoders13.5 Regularized Autoencoders13.5.1 Sparse Autoencoder13.5.2 Contractive Autoencoder13.5.3 Denoising Autoencoder13.5.4 Variational Autoencoder (VAE)
14. Natural Language Processing for Deep Learning
14.1 Motivation for Natural Language Processing14.1.1 Basics of Text Analytics14.2 Text Encoding14.2.1 One Hot Encoding14.2.2 Bag of Words (BoW)14.2.3 N-gram Approach14.2.4 TF-IDF Approach14.3 Word Embedding (or Text Vectorization)14.4 Pretrained Models for Word Embedding14.4.1 Word2Vec14.5 Case Studies for NLP 14.5.1 Sentiment Analysis14.5.2 Seq2seq Models
15. Transformer Architecture and Large Language Models
15.1 Overview of Transformers 15.2 Attention Mechanisms15.2.1 Context Creation15.2.2 Self-Attention 15.2.3 Multihead Attention15.3 Architecture of Transformers 15.3.1 Encoders15.3.2 Decoders15.3.3 Source and Target Representations15.3.4 Attention Layers 15.3.5 Add and Normalize15.3.6 Feedforward Networks15.3.7 Output15.4 Overview of Large Language Models15.4.1 Architecture of LLM15.4.2 Process of LLMs15.4.3 Pretraining and Fine Tuning of LLMs15.5 Google BERT15.5.1 BERT Architecture15.5.2 Pretraining of BERT15.6 ChatGPT15.6.1 Pretraining and Finetuning of GPT15.6.2 Prompt Engineering15.6.3 Zero-Shot and Few-Shot Learning
16. Generative AI and Generative Adversarial Networks
16.1 Overview of Generative AI16.2 Overview of GAN16.2.1 Principle of GAN16.2.2 Implementation of a Discriminator and Generator16.2.3 Architectures of Discriminator and Generator16.2.4 Adversarial Training of GAN16.2.5 Evaluation of GAN16.3 Popular GAN Architectures16.3.1 Stacked GAN16.3.2 Conditional GAN 16.3.3 Neural Style Transfer16.3.4 SuperResolution (SRGAN) 16.4 Diffusion Model16.4.1 Forward (Diffusion) Process16.4.2 Reverse Process16.4.3 DDPG Algorithm16.5 Variational Autoencoders16.6 Transformer-Based Image Generation
17. Boltzmann Machines and Deep Belief Networks
17.1 Introduction to Boltzmann Machines17.1.1 Restricted Boltzmann Machines17.2 Training of RBM17.2.1 RBM as a Classifier17.2.2 Stacked RBM17.3 Mathematical Interpretation of Training of RBM17.4 Deep Belief Network 17.4.1 DBN as a Classifier
18. Deep Reinforcement Learning
18.1 Overview of Reinforcement Learning18.2 Components of Reinforcement Learning18.3 Markov Decision Model 18.4 Dynamic Programming18.4.1 Value Iteration18.5 Model Free Methods18.6 Q-Learning18.6.1 Algorithm Q-Learning18.7 Deep Q-Network (DQN) 18.8 Policy Gradient Methods18.8.1 REINFORCE Algorithm
Appendix A: Installation of Keras
Appendix B: Keras Programming
Appendix C: Lab Manual
Bibliography
Index
Copyright

Content preview from Deep Learning

CHAPTER 18 Deep Reinforcement Learning

Reinforcement learning is the closest thing we have to a plausible model of how humans and animals learn.

— David Silver

Credits: © David Silver

Reinforcement learning is a branch of machine learning. Reinforcement learning is a type of machine learning where an agent interacts with the environment to learn. Learning is the interaction of the agent with the environment, and the agent learns by performing actions to maximize the rewards. The agent interacts with the environment, receives feedback, and adjusts the actions based on the rewards. In deep reinforcement learning, the deep neural networks are combined with reinforcement learning. There are many applications of reinforcement learning, such as ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9789367138663

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Deep Learning

by S.Sridhar

CHAPTER 18

Deep Reinforcement Learning

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.