Deep Learning with PyTorch

Deep Learning with PyTorch

by Vishnu Subramanian
Released February 2018
Publisher(s): Packt Publishing
ISBN: 9781788624336

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Build neural network models in text, vision and advanced analytics using PyTorch

About This Book

  • Learn PyTorch for implementing cutting-edge deep learning algorithms.
  • Train your neural networks for higher speed and flexibility and learn how to implement them in various scenarios;
  • Cover various advanced neural network architecture such as ResNet, Inception, DenseNet and more with practical examples;

Who This Book Is For

This book is for machine learning engineers, data analysts, data scientists interested in deep learning and are looking to explore implementing advanced algorithms in PyTorch. Some knowledge of machine learning is helpful but not a mandatory need. Working knowledge of Python programming is expected.

What You Will Learn

  • Use PyTorch for GPU-accelerated tensor computations
  • Build custom datasets and data loaders for images and test the models using torchvision and torchtext
  • Build an image classifier by implementing CNN architectures using PyTorch
  • Build systems that do text classification and language modeling using RNN, LSTM, and GRU
  • Learn advanced CNN architectures such as ResNet, Inception, Densenet, and learn how to use them for transfer learning
  • Learn how to mix multiple models for a powerful ensemble model
  • Generate new images using GAN's and generate artistic images using style transfer

In Detail

Deep learning powers the most intelligent systems in the world, such as Google Voice, Siri, and Alexa. Advancements in powerful hardware, such as GPUs, software frameworks such as PyTorch, Keras, Tensorflow, and CNTK along with the availability of big data have made it easier to implement solutions to problems in the areas of text, vision, and advanced analytics.

This book will get you up and running with one of the most cutting-edge deep learning libraries—PyTorch. PyTorch is grabbing the attention of deep learning researchers and data science professionals due to its accessibility, efficiency and being more native to Python way of development. You'll start off by installing PyTorch, then quickly move on to learn various fundamental blocks that power modern deep learning. You will also learn how to use CNN, RNN, LSTM and other networks to solve real-world problems. This book explains the concepts of various state-of-the-art deep learning architectures, such as ResNet, DenseNet, Inception, and Seq2Seq, without diving deep into the math behind them. You will also learn about GPU computing during the course of the book. You will see how to train a model with PyTorch and dive into complex neural networks such as generative networks for producing text and images.

By the end of the book, you'll be able to implement deep learning applications in PyTorch with ease.

Style and approach

An end-to-end guide that teaches you all about PyTorch and how to implement it in various scenarios.

Publisher resources

Download Example Code

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Deep Learning with PyTorch
  3. Dedication
  4. Packt Upsell
    1. Why subscribe?
    2. PacktPub.com
  5. Foreword
  6. Contributors
    1. About the author
    2. About the reviewer
    3. Packt is searching for authors like you
  7. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  8. Getting Started with Deep Learning Using PyTorch
    1. Artificial intelligence
      1. The history of AI
    2. Machine learning
      1. Examples of machine learning in real life
    3. Deep learning
      1. Applications of deep learning
      2. Hype associated with deep learning
      3. The history of deep learning
      4. Why now?
      5. Hardware availability
      6. Data and algorithms
      7. Deep learning frameworks
        1. PyTorch
    4. Summary
  9. Building Blocks of Neural Networks
    1. Installing PyTorch
    2. Our first neural network
      1. Data preparation
        1. Scalar (0-D tensors)
        2. Vectors (1-D tensors)
        3. Matrix (2-D tensors)
        4. 3-D tensors
        5. Slicing tensors
        6. 4-D tensors
        7. 5-D tensors
        8. Tensors on GPU
        9. Variables
      2. Creating data for our neural network
        1. Creating learnable parameters
        2. Neural network model
        3. Network implementation
        4. Loss function
        5. Optimize the neural network
      3. Loading data
        1. Dataset class
        2. DataLoader class
    3. Summary
  10. Diving Deep into Neural Networks
    1. Deep dive into the building blocks of neural networks
      1. Layers – fundamental blocks of neural networks
      2. Non-linear activations
        1. Sigmoid
        2. Tanh
        3. ReLU
        4. Leaky ReLU
      3. PyTorch non-linear activations
        1. The PyTorch way of building deep learning algorithms
        2. Model architecture for different machine learning problems
        3. Loss functions
        4. Optimizing network architecture
      4. Image classification using deep learning
        1. Loading data into PyTorch tensors
        2. Loading PyTorch tensors as batches
        3. Building the network architecture
        4. Training the model
    2. Summary
  11. Fundamentals of Machine Learning
    1. Three kinds of machine learning problems
      1. Supervised learning
      2. Unsupervised learning
      3. Reinforcement learning
    2. Machine learning glossary
    3. Evaluating machine learning models
      1. Training, validation, and test split
        1. Simple holdout validation
        2. K-fold validation
        3. K-fold validation with shuffling
          1. Data representativeness
          2. Time sensitivity
          3. Data redundancy
    4. Data preprocessing and feature engineering
      1. Vectorization
      2. Value normalization
      3. Handling missing values
      4. Feature engineering
    5. Overfitting and underfitting
      1. Getting more data
      2. Reducing the size of the network
      3. Applying weight regularization
      4. Dropout
      5. Underfitting
    6. Workflow of a machine learning project
      1. Problem definition and dataset creation
      2. Measure of success
      3. Evaluation protocol
      4. Prepare your data
      5. Baseline model
      6. Large model enough to overfit
      7. Applying regularization
      8. Learning rate picking strategies
    7. Summary
  12. Deep Learning for Computer Vision
    1. Introduction to neural networks
      1. MNIST – getting data
    2. Building a CNN model from scratch
      1. Conv2d
      2. Pooling
      3. Nonlinear activation – ReLU
      4. View
        1. Linear layer
      5. Training the model
      6. Classifying dogs and cats – CNN from scratch
      7. Classifying dogs and cats using transfer learning
    3. Creating and exploring a VGG16 model
      1. Freezing the layers
      2. Fine-tuning VGG16
      3. Training the VGG16 model
    4. Calculating pre-convoluted features
    5. Understanding what a CNN model learns
      1. Visualizing outputs from intermediate layers
    6. Visualizing weights of the CNN layer
    7. Summary
  13. Deep Learning with Sequence Data and Text
    1. Working with text data
      1. Tokenization
        1. Converting text into characters
        2. Converting text into words
        3. N-gram representation
      2. Vectorization
        1. One-hot encoding
        2. Word embedding
    2. Training word embedding by building a sentiment classifier
      1. Downloading IMDB data and performing text tokenization
        1. torchtext.data
        2. torchtext.datasets
      2. Building vocabulary
      3. Generate batches of vectors
      4. Creating a network model with embedding
      5. Training the model
    3. Using pretrained word embeddings
      1. Downloading the embeddings
      2. Loading the embeddings in the model
      3. Freeze the embedding layer weights
    4. Recursive neural networks
      1. Understanding how RNN works with an example
    5. LSTM
      1. Long-term dependency
      2. LSTM networks
        1. Preparing the data
        2. Creating batches
        3. Creating the network
        4. Training the model
    6. Convolutional network on sequence data
      1. Understanding one-dimensional convolution for sequence data
        1. Creating the network
        2. Training the model
    7. Summary
  14. Generative Networks
    1. Neural style transfer
      1. Loading the data
      2. Creating the VGG model
      3. Content loss
      4. Style loss
      5. Extracting the losses
      6. Creating loss function for each layers
      7. Creating the optimizer
      8. Training
    2. Generative adversarial networks
    3. Deep convolutional GAN
      1. Defining the generator network
        1. Transposed convolutions
        2. Batch normalization
        3. Generator
      2. Defining the discriminator network
      3. Defining loss and optimizer
      4. Training the discriminator
        1. Training the discriminator with real images
        2. Training the discriminator with fake images
      5. Training the generator network
      6. Training the complete network
      7. Inspecting the generated images
    4. Language modeling
      1. Preparing the data
      2. Generating the batches
        1. Batches
        2. Backpropagation through time
      3. Defining a model based on LSTM
      4. Defining the train and evaluate functions
      5. Training the model
    5. Summary
  15. Modern Network Architectures
    1. Modern network architectures
      1. ResNet
        1. Creating PyTorch datasets
        2. Creating loaders for training and validation
        3. Creating a ResNet model
        4. Extracting convolutional features
        5. Creating a custom PyTorch dataset class for the pre-convoluted features and loader
        6. Creating a simple linear model
        7. Training and validating the model
      2. Inception
        1. Creating an Inception model
        2. Extracting convolutional features using register_forward_hook
        3. Creating a new dataset for the convoluted features
        4. Creating a fully connected model
        5. Training and validating the model
    2. Densely connected convolutional networks – DenseNet
      1. DenseBlock
      2. DenseLayer
        1. Creating a DenseNet model
        2. Extracting DenseNet features
        3. Creating a dataset and loaders
        4. Creating a fully connected model and train
    3. Model ensembling
      1. Creating models
      2. Extracting the image features
      3. Creating a custom dataset along with data loaders
      4. Creating an ensembling model
      5. Training and validating the model
    4. Encoder-decoder architecture
      1. Encoder
      2. Decoder
    5. Summary
  16. What Next?
    1. What next?
    2. Overview
    3. Interesting ideas to explore
      1. Object detection
      2. Image segmentation
      3. OpenNMT in PyTorch
      4. Alien NLP
      5. fast.ai – making neural nets uncool again
      6. Open Neural Network Exchange
    4. How to keep yourself updated
    5. Summary
  17. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Deep Learning with PyTorch
  • Author(s): Vishnu Subramanian
  • Release date: February 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781788624336