book

Deep Learning with PyTorch Lightning

Name: Deep Learning with PyTorch Lightning
ISBN: 9781800561618

by Kunal Sawarkar, Dheeraj Arremsetty

April 2022

Intermediate to advanced

366 pages

7h 43m

English

Packt Publishing

Read now

Unlock full access

Deep Learning with PyTorch Lightning
ContributorsAbout the authorsAcknowledgementsAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the colour imagesConventions usedGet in touchShare Your Thoughts
Section 1: Kickstarting with PyTorch Lightning
Chapter 1: PyTorch Lightning Adventure
What makes PyTorch Lightning so special?The first one….So many frameworks? PyTorch versus TensorFlowA golden mean – PyTorch Lightning <pip install> – My Lightning adventure Understanding the key components of PyTorch Lightning DL pipelinePyTorch Lightning abstraction layersCrafting AI applications using PyTorch LightningImage recognition modelsTransfer learningNLP Transformer modelsLightning FlashTime series models with LSTMGenerative Adversarial Networks with AutoencodersSelf-Supervised models combining CNN and RNNSelf-Supervised models for contrastive learningDeploying and scoring modelsScaling models and productivity tipsFurther readingSummary
Chapter 2: Getting off the Ground with the First Deep Learning Model
Technical requirementsGetting started with Neural NetworksWhy Neural Networks?About the XOR operatorMLP architectureBuilding a Hello World MLP modelImporting librariesPreparing the dataConfiguring the modelTraining the modelLoading the modelMaking predictionsBuilding our first Deep Learning modelSo, what makes it deep?CNN architectureBuilding a CNN model for image recognitionImporting the packagesCollecting the dataPreparing the dataBuilding the modelTraining the modelEvaluating the accuracy of the modelModel improvement exercisesSummary
Chapter 3: Transfer Learning Using Pre-Trained Models
Technical requirementsGetting started with transfer learningAn image classifier using a pre-trained ResNet-50 architecturePreparing the data Extracting the datasetPre-processing the datasetLoading the datasetBuilding the modelTraining the modelEvaluating the accuracy of the model Text classification using BERT transformersCollecting the dataPreparing the datasetSetting up the DataLoader instancesBuilding the modelSetting up model training and testingTraining the modelEvaluating the modelSummary
Chapter 4: Ready-to-Cook Models from Lightning Flash
Technical requirementsGetting started with Lightning FlashFlash is as simple as 1-2-3Video classification using FlashSlow and SlowFast architecture Importing librariesLoading the datasetConfiguring the backbone Fine-tuning the modelMaking predictions using the modelAutomatic speech recognition using FlashInstalling Libraries Importing librariesLoading the datasetConfiguring the backbone Fine-tuning the modelSpeech prediction using the modelFurther learningSummary
Section 2: Solving using PyTorch Lightning
Chapter 5: Time Series Models
Technical requirementsIntroduction to time seriesTime series forecasting using Deep LearningGetting started with time series modelsTraffic volume forecasting using the LSTM time series modelDataset analysisFeature engineeringCreating a custom datasetConfiguring the LSTM model using PyTorch Lightning Setting up the optimizerTraining the modelMeasuring the training lossLoading the modelA prediction on the test datasetThe next stepsSummary
Chapter 6: Deep Generative Models
Technical requirementsGetting started with GAN modelsWhat is a GAN? Creating new food items using a GANLoading the datasetFeature engineering utility functionsThe discriminator modelThe generator modelThe generative adversarial modelTraining the GAN modelThe model output showing fake imagesCreating new butterfly species using a GANGAN training challengesCreating images using DCGAN Summary

Chapter 7: Semi-Supervised Learning
Technical requirementsGetting started with semi-supervised learningGoing through the CNN–RNN architectureGenerating captions for imagesDownloading the datasetAssembling the dataTraining the modelGenerating the captionNext stepsSummary
Chapter 8: Self-Supervised Learning
Technical requirementsGetting started with Self-Supervised LearningSo, what does it mean to be Self-Supervised?What is Contrastive Learning?SimCLR architectureHow does SimCLR work?SimCLR model for image recognitionCollecting the datasetSetting up data augmentationLoading the datasetTraining configuration Model trainingModel evaluationNext stepsSummary
Section 3: Advanced Topics
Chapter 9: Deploying and Scoring Models
Technical requirementsDeploying and scoring a Deep Learning model nativelyThe pickle (.PKL) model file formatDeploying our Deep Learning model Saving and loading model checkpointsDeploying and scoring a model using FlaskDeploying and scoring inter-portable modelsWhat is the ONNX format? Why does it matter?Saving and loading the ONNX modelDeploying and scoring the ONNX model using FlaskNext stepsFurther readingSummary
Chapter 10: Scaling and Managing Training
Technical RequirementsManaging training Saving model hyperparametersEfficient debugging Monitoring the training loss using TensorBoardScaling up trainingSpeeding up model training using a number of workersGPU/TPU trainingMixed precision training/16-bit trainingControlling trainingSaving model checkpoints when using the cloudChanging the default behavior of the checkpointing featureResuming training from a saved checkpointSaving downloaded and assembled data when using the cloudFurther readingSummary
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your Thoughts

Content preview from Deep Learning with PyTorch Lightning

Chapter 10: Scaling and Managing Training

So far, we have been on an exciting journey in the realm of Deep Learning (DL). We have learned how to recognize images, how to create new images or generate new texts, and how to train machines without fully labeled sets. It's an open secret that achieving good results for a DL model requires a massive amount of compute power, often requiring the help of a Graphics Processing Unit (GPU). We have come a long way since the early days of DL when data scientists had to manually distribute the training to each node of the GPU. PyTorch Lightning obfuscates most of the complexities associated with managing underlying hardware or pushing down training to the GPU.

In the earlier chapters, we have pushed down ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781800561618

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Deep Learning with PyTorch Lightning

by Kunal Sawarkar, Dheeraj Arremsetty

Chapter 10: Scaling and Managing Training

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.