Deep Learning with PyTorch Lightning

Deep Learning with PyTorch Lightning

by Kunal Sawarkar
Released April 2022
Publisher(s): Packt Publishing
ISBN: 9781800561618

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Build, train, deploy, and scale deep learning models quickly and accurately, improving your productivity using the lightweight PyTorch Wrapper

Key Features

  • Become well-versed with PyTorch Lightning architecture and learn how it can be implemented in various industry domains
  • Speed up your research using PyTorch Lightning by creating new loss functions, networks, and architectures
  • Train and build new algorithms for massive data using distributed training

Book Description

PyTorch Lightning lets researchers build their own Deep Learning (DL) models without having to worry about the boilerplate. With the help of this book, you'll be able to maximize productivity for DL projects while ensuring full flexibility from model formulation through to implementation. You'll take a hands-on approach to implementing PyTorch Lightning models to get up to speed in no time.

You'll start by learning how to configure PyTorch Lightning on a cloud platform, understand the architectural components, and explore how they are configured to build various industry solutions. Next, you'll build a network and application from scratch and see how you can expand it based on your specific needs, beyond what the framework can provide. The book also demonstrates how to implement out-of-box capabilities to build and train Self-Supervised Learning, semi-supervised learning, and time series models using PyTorch Lightning. As you advance, you'll discover how generative adversarial networks (GANs) work. Finally, you'll work with deployment-ready applications, focusing on faster performance and scaling, model scoring on massive volumes of data, and model debugging.

By the end of this PyTorch book, you'll have developed the knowledge and skills necessary to build and deploy your own scalable DL applications using PyTorch Lightning.

What you will learn

  • Customize models that are built for different datasets, model architectures, and optimizers
  • Understand how a variety of Deep Learning models from image recognition and time series to GANs, semi-supervised and self-supervised models can be built
  • Use out-of-the-box model architectures and pre-trained models using transfer learning
  • Run and tune DL models in a multi-GPU environment using mixed-mode precisions
  • Explore techniques for model scoring on massive workloads
  • Discover troubleshooting techniques while debugging DL models

Who this book is for

This deep learning book is for citizen data scientists and expert data scientists transitioning from other frameworks to PyTorch Lightning. This book will also be useful for deep learning researchers who are just getting started with coding for deep learning models using PyTorch Lightning. Working knowledge of Python programming and an intermediate-level understanding of statistics and deep learning fundamentals is expected.

Table of contents

  1. Deep Learning with PyTorch Lightning
  2. Contributors
  3. About the authors
  4. Acknowledgements
  5. About the reviewers
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Download the colour images
    6. Conventions used
    7. Get in touch
    8. Share Your Thoughts
  7. Section 1: Kickstarting with PyTorch Lightning
  8. Chapter 1: PyTorch Lightning Adventure
    1. What makes PyTorch Lightning so special?
      1. The first one….
      2. So many frameworks?
      3. PyTorch versus TensorFlow
      4. A golden mean – PyTorch Lightning
    2. <pip install> – My Lightning adventure
    3. Understanding the key components of PyTorch Lightning
      1. DL pipeline
      2. PyTorch Lightning abstraction layers
    4. Crafting AI applications using PyTorch Lightning
      1. Image recognition models
      2. Transfer learning
      3. NLP Transformer models
      4. Lightning Flash
      5. Time series models with LSTM
      6. Generative Adversarial Networks with Autoencoders
      7. Self-Supervised models combining CNN and RNN
      8. Self-Supervised models for contrastive learning
      9. Deploying and scoring models
      10. Scaling models and productivity tips
    5. Further reading
    6. Summary
  9. Chapter 2: Getting off the Ground with the First Deep Learning Model
    1. Technical requirements
    2. Getting started with Neural Networks
      1. Why Neural Networks?
      2. About the XOR operator
      3. MLP architecture
    3. Building a Hello World MLP model
      1. Importing libraries
      2. Preparing the data
      3. Configuring the model
      4. Training the model
      5. Loading the model
      6. Making predictions
    4. Building our first Deep Learning model
      1. So, what makes it deep?
      2. CNN architecture
    5. Building a CNN model for image recognition
      1. Importing the packages
      2. Collecting the data
      3. Preparing the data
      4. Building the model
      5. Training the model
      6. Evaluating the accuracy of the model
      7. Model improvement exercises
    6. Summary
  10. Chapter 3: Transfer Learning Using Pre-Trained Models
    1. Technical requirements
    2. Getting started with transfer learning
    3. An image classifier using a pre-trained ResNet-50 architecture
      1. Preparing the data
      2. Extracting the dataset
      3. Pre-processing the dataset
      4. Loading the dataset
      5. Building the model
      6. Training the model
      7. Evaluating the accuracy of the model
    4. Text classification using BERT transformers
      1. Collecting the data
      2. Preparing the dataset
      3. Setting up the DataLoader instances
      4. Building the model
      5. Setting up model training and testing
      6. Training the model
      7. Evaluating the model
    5. Summary
  11. Chapter 4: Ready-to-Cook Models from Lightning Flash
    1. Technical requirements
    2. Getting started with Lightning Flash
    3. Flash is as simple as 1-2-3
    4. Video classification using Flash
      1. Slow and SlowFast architecture
      2. Importing libraries
      3. Loading the dataset
      4. Configuring the backbone
      5. Fine-tuning the model
      6. Making predictions using the model
    5. Automatic speech recognition using Flash
      1. Installing Libraries
      2. Importing libraries
      3. Loading the dataset
      4. Configuring the backbone
      5. Fine-tuning the model
      6. Speech prediction using the model
    6. Further learning
    7. Summary
  12. Section 2: Solving using PyTorch Lightning
  13. Chapter 5: Time Series Models
    1. Technical requirements
    2. Introduction to time series
      1. Time series forecasting using Deep Learning
    3. Getting started with time series models
    4. Traffic volume forecasting using the LSTM time series model
      1. Dataset analysis
      2. Feature engineering
      3. Creating a custom dataset
      4. Configuring the LSTM model using PyTorch Lightning
      5. Setting up the optimizer
      6. Training the model
      7. Measuring the training loss
      8. Loading the model
      9. A prediction on the test dataset
      10. The next steps
    5. Summary
  14. Chapter 6: Deep Generative Models
    1. Technical requirements
    2. Getting started with GAN models
      1. What is a GAN?
    3. Creating new food items using a GAN
      1. Loading the dataset
      2. Feature engineering utility functions
      3. The discriminator model
      4. The generator model
      5. The generative adversarial model
      6. Training the GAN model
      7. The model output showing fake images
    4. Creating new butterfly species using a GAN
    5. GAN training challenges
    6. Creating images using DCGAN
    7. Summary
  15. Chapter 7: Semi-Supervised Learning
    1. Technical requirements
    2. Getting started with semi-supervised learning
    3. Going through the CNN–RNN architecture
    4. Generating captions for images
      1. Downloading the dataset
      2. Assembling the data
      3. Training the model
      4. Generating the caption
      5. Next steps
    5. Summary
  16. Chapter 8: Self-Supervised Learning
    1. Technical requirements
    2. Getting started with Self-Supervised Learning
      1. So, what does it mean to be Self-Supervised?
    3. What is Contrastive Learning?
    4. SimCLR architecture
      1. How does SimCLR work?
    5. SimCLR model for image recognition
      1. Collecting the dataset
      2. Setting up data augmentation
      3. Loading the dataset
      4. Training configuration
      5. Model training
      6. Model evaluation
      7. Next steps
    6. Summary
  17. Section 3: Advanced Topics
  18. Chapter 9: Deploying and Scoring Models
    1. Technical requirements
    2. Deploying and scoring a Deep Learning model natively
      1. The pickle (.PKL) model file format
      2. Deploying our Deep Learning model
      3. Saving and loading model checkpoints
      4. Deploying and scoring a model using Flask
    3. Deploying and scoring inter-portable models
      1. What is the ONNX format? Why does it matter?
      2. Saving and loading the ONNX model
      3. Deploying and scoring the ONNX model using Flask
    4. Next steps
    5. Further reading
    6. Summary
  19. Chapter 10: Scaling and Managing Training
    1. Technical Requirements
    2. Managing training
      1. Saving model hyperparameters
      2. Efficient debugging
      3. Monitoring the training loss using TensorBoard
    3. Scaling up training
      1. Speeding up model training using a number of workers
      2. GPU/TPU training
      3. Mixed precision training/16-bit training
    4. Controlling training
      1. Saving model checkpoints when using the cloud
      2. Changing the default behavior of the checkpointing feature
      3. Resuming training from a saved checkpoint
      4. Saving downloaded and assembled data when using the cloud
    5. Further reading
    6. Summary
    7. Why subscribe?
  20. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Share Your Thoughts

Product information

  • Title: Deep Learning with PyTorch Lightning
  • Author(s): Kunal Sawarkar
  • Release date: April 2022
  • Publisher(s): Packt Publishing
  • ISBN: 9781800561618