O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Programming PyTorch for Deep Learning

Programming PyTorch for Deep Learning

by Ian Pointer
Publisher: O'Reilly Media, Inc.
Release Date: September 2019
ISBN: 9781492045342
View table of contents
Start reading

Book Description

Deep learning is changing everything. This machine learning method has already surpassed traditional computer vision techniques, and the same is happening with NLP. If you're looking to bring deep learning into your domain, this practical book will bring you up to speed on key concepts using Facebook's PyTorch framework.

Once author Ian Pointer helps you set up PyTorch on a cloud-based environment, you'll learn how use the framework to create neural architectures for performing operations on images, sound, text, and other types of data. By the end of the book, you'll be able to create neural networks and train them on multiple types of data.

  • Learn how to deploy deep learning models to production
  • Explore PyTorch use cases in companies other than Facebook
  • Learn how to apply transfer learning to images
  • Apply cutting-edge NLP techniques using a model trained on Wikipedia

Table of Contents

  1. Preface
    1. Deep Learning in the World Today
    2. But What Is Deep Learning Exactly, and Do I Need a PhD to Understand It?
    3. PyTorch
      1. What About TensorFlow?
    4. Conventions Used in This Book
    5. Using Code Examples
    6. O’Reilly Online Learning
    7. How to Contact Us
    8. Acknowledgments
  2. 1. Getting Started with PyTorch
    1. Building a Custom Deep Learning Machine
      1. GPU
      2. CPU/Motherboard
      3. RAM
      4. Storage
    2. Deep Learning in the Cloud
      1. Google Colaboratory
      2. Cloud Providers
      3. Which Cloud Provider Should I Use?
    3. Using Jupyter Notebook
    4. Installing PyTorch from Scratch
      1. Download CUDA
      2. Anaconda
      3. Finally, PyTorch! (and Jupyter Notebook)
    5. Tensors
      1. Tensor Operations
      2. Tensor Broadcasting
    6. Conclusion
    7. Further Reading
  3. 2. Image Classification with PyTorch
    1. Our Classification Problem
    2. Traditional Challenges
      1. But First, Data
      2. PyTorch and Data Loaders
      3. Building a Training Dataset
      4. Building Validation and Test Datasets
    3. Finally, a Neural Network!
      1. Activation Functions
      2. Creating a Network
      3. Loss Functions
      4. Optimizing
    4. Training
      1. Making It Work on the GPU
    5. Putting It All Together
      1. Making Predictions
      2. Model Saving
    6. Conclusion
    7. Further Reading
  4. 3. Convolutional Neural Networks
    1. Our First Convolutional Model
      1. Convolutions
      2. Pooling
      3. Dropout
    2. History of CNN Architectures
      1. AlexNet
      2. Inception/GoogLeNet
      3. VGG
      4. ResNet
      5. Other Architectures Are Available!
    3. Using Pretrained Models in PyTorch
      1. Examining a Model’s Structure
      2. BatchNorm
      3. Which Model Should You Use?
    4. One-Stop Shopping for Models: PyTorch Hub
    5. Conclusion
    6. Further Reading
  5. 4. Transfer Learning and Other Tricks
    1. Transfer Learning with ResNet
    2. Finding That Learning Rate
    3. Differential Learning Rates
    4. Data Augmentation
      1. Torchvision Transforms
      2. Color Spaces and Lambda Transforms
      3. Custom Transform Classes
      4. Start Small and Get Bigger!
    5. Ensembles
    6. Conclusion
    7. Further Reading
  6. 5. Text Classification
    1. Recurrent Neural Networks
    2. Long Short-Term Memory Networks
      1. Gated Recurrent Units
      2. biLSTM
    3. Embeddings
    4. torchtext
      1. Getting Our Data: Tweets!
      2. Defining Fields
      3. Building a Vocabulary
      4. Creating Our Model
      5. Updating the Training Loop
      6. Classifying Tweets
    5. Data Augmentation
      1. Random Insertion
      2. Random Deletion
      3. Random Swap
      4. Back Translation
      5. Augmentation and torchtext
      6. Transfer Learning?
    6. Conclusion
    7. Further Reading
  7. 6. A Journey into Sound
    1. Sound
    2. The ESC-50 Dataset
      1. Obtaining the Dataset
      2. Playing Audio in Jupyter
    3. Exploring ESC-50
      1. SoX and LibROSA
      2. torchaudio
      3. Building an ESC-50 Dataset
    4. A CNN Model for ESC-50
    5. This Frequency Is My Universe
      1. Mel Spectrograms
      2. A New Dataset
      3. A Wild ResNet Appears
      4. Finding a Learning Rate
    6. Audio Data Augmentation
      1. torchaudio Transforms
      2. SoX Effect Chains
      3. SpecAugment
    7. Further Experiments
    8. Conclusion
    9. Further Reading
  8. 7. Debugging PyTorch Models
    1. It’s 3 a.m. What Is Your Data Doing?
    2. TensorBoard
      1. Installing TensorBoard
      2. Sending Data to TensorBoard
      3. PyTorch Hooks
      4. Plotting Mean and Standard Deviation
      5. Class Activation Mapping
    3. Flame Graphs
      1. Installing py-spy
      2. Reading Flame Graphs
      3. Fixing a Slow Transformation
    4. Debugging GPU Issues
      1. Checking Your GPU
      2. Gradient Checkpointing
    5. Conclusion
    6. Further Reading
  9. 8. PyTorch in Production
    1. Model Serving
      1. Building a Flask Service
      2. Setting Up the Model Parameters
      3. Building the Docker Container
      4. Local Versus Cloud Storage
      5. Logging and Telemetry
    2. Deploying on Kubernetes
      1. Setting Up on Google Kubernetes Engine
      2. Creating a k8s Cluster
      3. Scaling Services
      4. Updates and Cleaning Up
    3. TorchScript
      1. Tracing
      2. Scripting
      3. TorchScript Limitations
    4. Working with libTorch
      1. Obtaining libTorch and Hello World
      2. Importing a TorchScript Model
    5. Conclusion
    6. Further Reading
  10. 9. PyTorch in the Wild
    1. Data Augmentation: Mixed and Smoothed
      1. mixup
      2. Label Smoothing
    2. Computer, Enhance!
      1. Introduction to Super-Resolution
      2. An Introduction to GANs
      3. The Forger and the Critic
      4. Training a GAN
      5. The Dangers of Mode Collapse
      6. ESRGAN
    3. Further Adventures in Image Detection
      1. Object Detection
      2. Faster R-CNN and Mask R-CNN
    4. Adversarial Samples
      1. Black-Box Attacks
      2. Defending Against Adversarial Attacks
    5. More Than Meets the Eye: The Transformer Architecture
      1. Paying Attention
      2. Attention Is All You Need
      3. BERT
      4. FastBERT
      5. GPT-2
      6. Generating Text with GPT-2
      7. ULMFiT
      8. What to Use?
    6. Conclusion
    7. Further Reading
  11. Index