book

Deep Learning with PyTorch Lightning

Name: Deep Learning with PyTorch Lightning
ISBN: 9781800561618

by Kunal Sawarkar, Dheeraj Arremsetty

April 2022

Intermediate to advanced

366 pages

7h 43m

English

Packt Publishing

Read now

Unlock full access

Deep Learning with PyTorch Lightning
ContributorsAbout the authorsAcknowledgementsAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the colour imagesConventions usedGet in touchShare Your Thoughts
Section 1: Kickstarting with PyTorch Lightning
Chapter 1: PyTorch Lightning Adventure
What makes PyTorch Lightning so special?The first one….So many frameworks? PyTorch versus TensorFlowA golden mean – PyTorch Lightning <pip install> – My Lightning adventure Understanding the key components of PyTorch Lightning DL pipelinePyTorch Lightning abstraction layersCrafting AI applications using PyTorch LightningImage recognition modelsTransfer learningNLP Transformer modelsLightning FlashTime series models with LSTMGenerative Adversarial Networks with AutoencodersSelf-Supervised models combining CNN and RNNSelf-Supervised models for contrastive learningDeploying and scoring modelsScaling models and productivity tipsFurther readingSummary
Chapter 2: Getting off the Ground with the First Deep Learning Model
Technical requirementsGetting started with Neural NetworksWhy Neural Networks?About the XOR operatorMLP architectureBuilding a Hello World MLP modelImporting librariesPreparing the dataConfiguring the modelTraining the modelLoading the modelMaking predictionsBuilding our first Deep Learning modelSo, what makes it deep?CNN architectureBuilding a CNN model for image recognitionImporting the packagesCollecting the dataPreparing the dataBuilding the modelTraining the modelEvaluating the accuracy of the modelModel improvement exercisesSummary
Chapter 3: Transfer Learning Using Pre-Trained Models
Technical requirementsGetting started with transfer learningAn image classifier using a pre-trained ResNet-50 architecturePreparing the data Extracting the datasetPre-processing the datasetLoading the datasetBuilding the modelTraining the modelEvaluating the accuracy of the model Text classification using BERT transformersCollecting the dataPreparing the datasetSetting up the DataLoader instancesBuilding the modelSetting up model training and testingTraining the modelEvaluating the modelSummary
Chapter 4: Ready-to-Cook Models from Lightning Flash
Technical requirementsGetting started with Lightning FlashFlash is as simple as 1-2-3Video classification using FlashSlow and SlowFast architecture Importing librariesLoading the datasetConfiguring the backbone Fine-tuning the modelMaking predictions using the modelAutomatic speech recognition using FlashInstalling Libraries Importing librariesLoading the datasetConfiguring the backbone Fine-tuning the modelSpeech prediction using the modelFurther learningSummary
Section 2: Solving using PyTorch Lightning
Chapter 5: Time Series Models
Technical requirementsIntroduction to time seriesTime series forecasting using Deep LearningGetting started with time series modelsTraffic volume forecasting using the LSTM time series modelDataset analysisFeature engineeringCreating a custom datasetConfiguring the LSTM model using PyTorch Lightning Setting up the optimizerTraining the modelMeasuring the training lossLoading the modelA prediction on the test datasetThe next stepsSummary
Chapter 6: Deep Generative Models
Technical requirementsGetting started with GAN modelsWhat is a GAN? Creating new food items using a GANLoading the datasetFeature engineering utility functionsThe discriminator modelThe generator modelThe generative adversarial modelTraining the GAN modelThe model output showing fake imagesCreating new butterfly species using a GANGAN training challengesCreating images using DCGAN Summary

Chapter 7: Semi-Supervised Learning
Technical requirementsGetting started with semi-supervised learningGoing through the CNN–RNN architectureGenerating captions for imagesDownloading the datasetAssembling the dataTraining the modelGenerating the captionNext stepsSummary
Chapter 8: Self-Supervised Learning
Technical requirementsGetting started with Self-Supervised LearningSo, what does it mean to be Self-Supervised?What is Contrastive Learning?SimCLR architectureHow does SimCLR work?SimCLR model for image recognitionCollecting the datasetSetting up data augmentationLoading the datasetTraining configuration Model trainingModel evaluationNext stepsSummary
Section 3: Advanced Topics
Chapter 9: Deploying and Scoring Models
Technical requirementsDeploying and scoring a Deep Learning model nativelyThe pickle (.PKL) model file formatDeploying our Deep Learning model Saving and loading model checkpointsDeploying and scoring a model using FlaskDeploying and scoring inter-portable modelsWhat is the ONNX format? Why does it matter?Saving and loading the ONNX modelDeploying and scoring the ONNX model using FlaskNext stepsFurther readingSummary
Chapter 10: Scaling and Managing Training
Technical RequirementsManaging training Saving model hyperparametersEfficient debugging Monitoring the training loss using TensorBoardScaling up trainingSpeeding up model training using a number of workersGPU/TPU trainingMixed precision training/16-bit trainingControlling trainingSaving model checkpoints when using the cloudChanging the default behavior of the checkpointing featureResuming training from a saved checkpointSaving downloaded and assembled data when using the cloudFurther readingSummary
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your Thoughts

Content preview from Deep Learning with PyTorch Lightning

Chapter 3: Transfer Learning Using Pre-Trained Models

Deep learning models become more accurate the more data they have for training. The most spectacular Deep Learning models, such as ImageNet, are trained on millions of images and often require a massive amount of computing power. To put things into perspective, the amount of power used to train OpenAI's GPT3 model could power an entire city. Unsurprisingly, the cost of training such Deep Learning models from scratch is prohibitive for most projects.

This begs the question: do we really need to train a Deep Learning model from scratch each time? One way of getting around this problem, rather than training Deep Learning models from scratch, is to borrow representations from an already trained ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781800561618

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Deep Learning with PyTorch Lightning

by Kunal Sawarkar, Dheeraj Arremsetty

Chapter 3: Transfer Learning Using Pre-Trained Models

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.