Deep Learning Essentials

Deep Learning Essentials

by Wei Di, Anurag Bhardwaj, Jianing Wei
Released January 2018
Publisher(s): Packt Publishing
ISBN: 9781785880360

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Get to grips with the essentials of deep learning by leveraging the power of Python

About This Book

  • Your one-stop solution to get started with the essentials of deep learning and neural network modeling
  • Train different kinds of neural networks to tackle various problems in Natural Language Processing, computer vision, speech recognition, and more
  • Covers popular Python libraries such as Tensorflow, Keras, and more, along with tips on training, deploying and optimizing your deep learning models in the best possible manner

Who This Book Is For

Aspiring data scientists and machine learning experts who have limited or no exposure to deep learning will find this book to be very useful. If you are looking for a resource that gets you up and running with the fundamentals of deep learning and neural networks, this book is for you. As the models in the book are trained using the popular Python-based libraries such as Tensorflow and Keras, it would be useful to have sound programming knowledge of Python.

What You Will Learn

  • Get to grips with the core concepts of deep learning and neural networks
  • Set up deep learning library such as TensorFlow
  • Fine-tune your deep learning models for NLP and Computer Vision applications
  • Unify different information sources, such as images, text, and speech through deep learning
  • Optimize and fine-tune your deep learning models for better performance
  • Train a deep reinforcement learning model that plays a game better than humans
  • Learn how to make your models get the best out of your GPU or CPU

In Detail

Deep Learning a trending topic in the field of Artificial Intelligence today and can be considered to be an advanced form of machine learning, which is quite tricky to master.

This book will help you take your first steps in training efficient deep learning models and applying them in various practical scenarios. You will model, train, and deploy different kinds of neural networks such as Convolutional Neural Network, Recurrent Neural Network, and will see some of their applications in real-world domains including computer vision, natural language processing, speech recognition, and so on. You will build practical projects such as chatbots, implement reinforcement learning to build smart games, and develop expert systems for image captioning and processing. Popular Python library such as TensorFlow is used in this book to build the models. This book also covers solutions for different problems you might come across while training models, such as noisy datasets, small datasets, and more.

This book does not assume any prior knowledge of deep learning. By the end of this book, you will have a firm understanding of the basics of deep learning and neural network modeling, along with their practical applications.

Style and approach

This step-by-step guide is filled with real-world practical examples and use cases to solve various deep learning problems.

Table of contents

  1. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  2. Why Deep Learning?
    1. What is AI and deep learning?
    2. The history and rise of deep learning
    3. Why deep learning?
      1. Advantages over traditional shallow methods
      2. Impact of deep learning
    4. The motivation of deep architecture
      1. The neural viewpoint
      2. The representation viewpoint
        1. Distributed feature representation
        2. Hierarchical feature representation
    5. Applications
      1. Lucrative applications
      2. Success stories
      3. Deep learning for business
    6. Future potential and challenges
    7. Summary
  3. Getting Yourself Ready for Deep Learning
    1. Basics of linear algebra
      1. Data representation
      2. Data operations
      3. Matrix properties
    2. Deep learning with GPU
      1. Deep learning hardware guide
        1. CPU cores
        2. CPU cache size
        3. RAM size
        4. Hard drive
        5. Cooling systems
    3. Deep learning software frameworks
      1. TensorFlow – a deep learning library
      2. Caffe
      3. MXNet
      4. Torch
      5. Theano
      6. Microsoft Cognitive Toolkit
      7. Keras
      8. Framework comparison
    4. Setting up deep learning on AWS
      1. Setup from scratch
      2. Setup using Docker
    5. Summary
  4. Getting Started with Neural Networks
    1. Multilayer perceptrons
      1. The input layer
      2. The output layer
      3. Hidden layers
      4. Activation functions
        1. Sigmoid or logistic function
        2. Tanh or hyperbolic tangent function
        3. ReLU
        4. Leaky ReLU and maxout
        5. Softmax
        6. Choosing the right activation function
    2. How a network learns
      1. Weight initialization
      2. Forward propagation
      3. Backpropagation
        1. Calculating errors
        2. Backpropagation
        3. Updating the network
        4. Automatic differentiation
      4. Vanishing and exploding gradients
      5. Optimization algorithms
      6. Regularization
    3. Deep learning models
      1. Convolutional Neural Networks
        1. Convolution
        2. Pooling/subsampling
        3. Fully connected layer
        4. Overall
      2. Restricted Boltzmann Machines
        1. Energy function
        2. Encoding and decoding
        3. Contrastive divergence (CD-k)
        4. Stacked/continuous RBM
        5. RBM versus Boltzmann Machines
      3. Recurrent neural networks (RNN/LSTM)
        1. Cells in RNN and unrolling
        2. Backpropagation through time
        3. Vanishing gradient and LTSM
        4. Cells and gates in LTSM
          1. Step 1 – The forget gate
          2. Step 2 – Updating memory/cell state
          3. Step 3 – The output gate
    4. Practical examples
      1. TensorFlow setup and key concepts
      2. Handwritten digits recognition
    5. Summary
  5. Deep Learning in Computer Vision
    1. Origins of CNNs
    2. Convolutional Neural Networks
      1. Data transformations
        1. Input preprocessing
        2. Data augmentation
      2. Network layers
        1. Convolution layer
        2. Pooling or subsampling layer
        3. Fully connected or dense layer
      3. Network initialization
      4. Regularization
      5. Loss functions
      6. Model visualization
      7. Handwritten digit classification example
    3. Fine-tuning CNNs
    4. Popular CNN architectures
      1. AlexNet
      2. Visual Geometry Group
      3. GoogLeNet
      4. ResNet
    5. Summary
  6. NLP - Vector Representation
    1. Traditional NLP
      1. Bag of words
      2. Weighting the terms tf-idf
    2. Deep learning NLP
      1. Motivation and distributed representation
      2. Word embeddings
        1. Idea of word embeddings
        2. Advantages of distributed representation
        3. Problems of distributed representation
        4. Commonly used pre-trained word embeddings
      3. Word2Vec
        1. Basic idea of Word2Vec
        2. The word windows
        3. Generating training data
        4. Negative sampling
        5. Hierarchical softmax
        6. Other hyperparameters
        7. Skip-Gram model
          1. The input layer
          2. The hidden layer
          3. The output layer
          4. The loss function
        8. Continuous Bag-of-Words model
        9. Training a Word2Vec using TensorFlow
        10. Using existing pre-trained Word2Vec embeddings
          1. Word2Vec from Google News
          2. Using the pre-trained Word2Vec embeddings
      4. Understanding GloVe
      5. FastText
    3. Applications
      1. Example use cases
      2. Fine-tuning
    4. Summary
  7. Advanced Natural Language Processing
    1. Deep learning for text
      1. Limitations of neural networks
    2. Recurrent neural networks
      1. RNN architectures
      2. Basic RNN model
      3. Training RNN is tough
    3. Long short-term memory network
      1. LSTM implementation with tensorflow
    4. Applications
      1. Language modeling
      2. Sequence tagging
      3. Machine translation
        1. Seq2Seq inference
      4. Chatbots
    5. Summary
  8. Multimodality
    1. What is multimodality learning?
    2. Challenges of multimodality learning
      1. Representation
      2. Translation
      3. Alignment
      4. Fusion
      5. Co-learning
    3. Image captioning
      1. Show and tell
        1. Encoder
        2. Decoder
        3. Training
        4. Testing/inference
        5. Beam Search
      2. Other types of approaches
      3. Datasets
      4. Evaluation
        1. BLEU
        2. ROUGE
        3. METEOR
        4. CIDEr
        5. SPICE
        6. Rank position
      5. Attention models
        1. Attention in NLP
        2. Attention in computer vision
          1. The difference between hard attention and soft attention
    4. Visual question answering
    5. Multi-source based self-driving
    6. Summary
  9. Deep Reinforcement Learning
    1. What is reinforcement learning (RL)?
      1. Problem setup
      2. Value learning-based algorithms
      3. Policy search-based algorithms
      4. Actor-critic-based algorithms
    2. Deep reinforcement learning
      1. Deep Q-network (DQN)
        1. Experience replay
        2. Target network
        3. Reward clipping
      2. Double-DQN
        1. Prioritized experience delay
      3. Dueling DQN
    3. Implementing reinforcement learning
      1. Simple reinforcement learning example
      2. Reinforcement learning with Q-learning example
    4. Summary
  10. Deep Learning Hacks
    1. Massaging your data
      1. Data cleaning
      2. Data augmentation
      3. Data normalization
    2. Tricks in training
      1. Weight initialization
        1. All-zero
        2. Random initialization
        3. ReLU initialization
        4. Xavier initialization
      2. Optimization
        1. Learning rate
        2. Mini-batch
        3. Clip gradients
      3. Choosing the loss function
        1. Multi-class classification
        2. Multi-class multi-label classification
        3. Regression
        4. Others
      4. Preventing overfitting
        1. Batch normalization
        2. Dropout
        3. Early stopping
        4. Fine-tuning
    3. Fine-tuning
      1. When to use fine-tuning
      2. When not to use fine-tuning
      3. Tricks and techniques
      4. List of pre-trained models
    4. Model compression
    5. Summary
  11. Deep Learning Trends
    1. Recent models for deep learning
      1. Generative Adversarial Networks
      2. Capsule networks
    2. Novel applications
      1. Genomics
      2. Predictive medicine
      3. Clinical imaging
      4. Lip reading
      5. Visual reasoning
      6. Code synthesis
    3. Summary
  12. Other Books You May Enjoy
    1. Leave a review – let other readers know what you think

Product information

  • Title: Deep Learning Essentials
  • Author(s): Wei Di, Anurag Bhardwaj, Jianing Wei
  • Release date: January 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781785880360