book

Deep Learning with PyTorch, Second Edition

Name: Deep Learning with PyTorch, Second Edition
ISBN: 9781633438859

by Thomas Viehmann, Eli Stevens, Luca Pietro Giovanni Antiga, Howard Huang

March 2026

Intermediate to advanced

544 pages

15h 56m

English

Manning Publications

Audiobook available

Read now

Unlock full access

Deep Learning with Pytorch, Second Edition
copyright
contents
Praise for the first edition
dedication
preface
acknowledgments
about this book
about the authors
about the cover illustration

Part 1 Core PyTorch
1 Introducing deep learning and the PyTorch library
1.1 What is deep learning?1.2 The shift from machine learning to deep learning1.3 What to expect1.4 Why PyTorch?1.4.1 The deep learning competitive landscape1.5 How PyTorch supports deep learning projects1.6 Hardware and software requirements1.6.1 Using Jupyter Notebooks1.7 Exercises
2 Pretrained networks
2.1 A pretrained network that recognizes the subject of an image2.1.1 Obtaining a pretrained network for image recognition2.1.2 AlexNet2.1.3 The Vision Transformer2.1.4 Ready, set, almost run2.1.5 Run!2.2 Generating and editing images2.2.1 The inpainting process2.2.2 A network that turns horses into zebras2.3 Model Zoo: Hugging Face2.4 A pretrained network that describes scenes2.4.1 BLIP in action2.5 Conclusion2.6 Exercises
3 It starts with a tensor
3.1 The world as floating-point numbers3.2 Tensors: Multidimensional arrays3.2.1 From Python lists to PyTorch tensors3.2.2 Constructing our first tensors3.2.3 The essence of tensors3.3 Indexing tensors3.4 Broadcasting3.5 Named tensors3.6 Tensor element types3.6.1 Specifying the numeric type with dtype3.6.2 A dtype for every occasion3.6.3 Managing a tensor’s dtype attribute3.7 The tensor API3.8 Tensors: Scenic views of storage3.8.1 Indexing into storage3.8.2 Modifying stored values: In-place operations3.9 Tensor metadata: Size, offset, and stride3.9.1 Views of another tensor’s storage3.9.2 Transposing without copying3.9.3 Transposing in higher dimensions3.9.4 Contiguous tensors3.10 Moving tensors to the GPU3.10.1 Managing a tensor’s device attribute3.11 NumPy interoperability3.12 Generalized tensors are tensors, too3.13 Serializing tensors3.13.1 Serializing to HDF5 with h5py3.14 Conclusion3.15 Exercises
4 Real-world data representation using tensors
4.1 Working with images4.1.1 Adding color channels4.1.2 Loading an image file4.1.3 Changing the layout4.1.4 Normalizing the data4.2 3D images: Volumetric data4.2.1 Loading a specialized format4.3 Representing tabular data4.3.1 Using a real-world dataset4.3.2 Loading a wine data tensor4.3.3 Representing scores4.3.4 One-hot encoding4.3.5 When to categorize4.3.6 Finding thresholds4.4 Working with time series4.4.1 Adding a time dimension4.4.2 Shaping the data by time period4.4.3 Ready for training4.5 Representing text4.5.1 Converting text to numbers4.5.2 One-hot-encoding characters4.5.3 One-hot encoding whole words4.5.4 Text embeddings4.5.5 Text embeddings as a blueprint4.6 Conclusion4.7 Exercises
5 The mechanics of learning
5.1 A timeless lesson in modeling5.2 Learning is just parameter estimation5.2.1 A hot problem5.2.2 Gathering some data5.2.3 Visualizing the data5.2.4 Choosing a linear model as a first try5.3 Less loss is what we want5.3.1 From problem back to PyTorch5.4 Down along the gradient5.4.1 Decreasing loss5.4.2 Getting analytical5.4.3 Iterating to fit the model5.4.4 Normalizing inputs5.4.5 Visualizing (again)5.5 PyTorch’s autograd: Backpropagating all things5.5.1 Computing the gradient automatically5.5.2 Optimizers à la carte5.5.3 Training, validation, and overfitting5.5.4 Training set5.5.5 Autograd nits and switching it off5.6 Conclusion5.7 Exercises
6 Using a neural network to fit the data
6.1 Artificial neurons6.1.1 Composing a multilayer network6.1.2 Understanding the error function6.1.3 Adding nonlinearity with activation functions6.1.4 More activation functions6.1.5 Choosing the best activation function6.1.6 What learning means for a neural network6.2 The PyTorch nn module6.2.1 Using nn.Module as a callable6.2.2 Returning to the linear model6.3 Finally, a neural network6.3.1 Replacing the linear model6.3.2 Inspecting the parameters6.3.3 Comparing to the linear model6.4 Conclusion6.5 Exercises
7 Telling birds from airplanes: Learning from images
7.1 A dataset of tiny images7.1.1 Downloading CIFAR-107.1.2 The Dataset class7.1.3 Dataset transforms7.1.4 Normalizing data7.2 Distinguishing birds from airplanes7.2.1 Building the dataset7.2.2 A fully connected model7.2.3 Output of a classifier7.2.4 Representing the output as probabilities7.2.5 Training the classifier7.2.6 The limits of going fully connected7.3 Conclusion7.4 Exercises
8 Using convolutions to generalize
8.1 The case for convolutions8.1.1 What convolutions do8.2 Convolutions in action8.2.1 Padding the boundary8.2.2 Detecting features with convolutions8.2.3 Looking further with depth and pooling8.2.4 Putting it all together for our network8.3 Subclassing nn.Module8.3.1 Our network as an nn.Module8.3.2 How PyTorch keeps track of parameters and submodules8.3.3 The functional API8.4 Training our convolutional neural network8.4.1 Measuring accuracy8.4.2 Saving and loading our model8.4.3 Training on the GPU8.5 Model design8.5.1 Adding memory capacity: Width8.5.2 Helping our model to converge and generalize: Regularization8.5.3 Going deeper to learn more complex structures: Depth8.5.4 Comparing the designs from this section8.5.5 It’s already outdated8.6 Conclusion8.7 Exercises
Part 2 Practical deep learning applications
9 How transformers work
9.1 A motivating example: Generating names character by character9.2 Self-supervised learning9.2.1 Limits of the bigram model9.3 Generating our training data9.4 Embeddings and linear layers9.4.1 Visualizing embeddings9.5 Attention9.5.1 Dot product self-attention9.5.2 Scaled dot product causal self-attention9.6 Transformers9.6.1 The decoder9.7 Other Transformer architectures9.7.1 The encoder9.7.2 The encoder-decoder9.8 Tokenization9.8.1 Generating sentences9.9 The Vision Transformer9.10 Conclusion9.11 Exercises
10 Diffusion models for images
10.1 History of VAEs and GANs10.2 Motivator for diffusion models10.3 Diffusion in detail10.4 Setting up the data10.5 The forward process10.6 Training10.6.1 Loss10.7 Reversing diffusion (how to sample)10.8 Conclusion10.9 Exercises
11 Using PyTorch to fight cancer
11.1 Introduction to the use case11.2 Preparing for a large-scale project11.3 What is a CT scan, exactly?11.4 The project: An end-to-end detector for lung cancer11.4.1 Why can’t we just throw data at a neural network until it works?11.4.2 Our data source: The LUNA Grand Challenge11.4.3 Downloading the LUNA data11.5 Conclusion
12 Combining data sources into a unified dataset
12.1 Raw CT data files12.2 Parsing LUNA’s annotation data12.2.1 Training and validation sets12.2.2 Unifying our annotation and candidate data12.3 Loading individual CT scans12.3.1 Hounsfield Units12.4 Locating a nodule using the patient coordinate system12.4.1 The patient coordinate system12.4.2 CT scan shape and voxel sizes12.4.3 Converting between millimeters and voxel addresses12.4.4 Extracting a nodule from a CT scan12.5 Straightforward dataset implementation12.5.1 Caching candidate arrays with the getCtRawCandidate function12.5.2 Constructing our dataset in LunaDataset.__init__12.5.3 A training/validation split12.5.4 Rendering the data12.6 Conclusion12.7 Exercises
13 Training a classification model to detect suspected tumors
13.1 A foundational model and training loop13.2 The main entry point for our application13.3 Pretraining setup and initialization13.3.1 Initializing the model and optimizer13.3.2 Care and feeding of data loaders13.4 Our first-pass neural network design13.4.1 The core convolutions13.4.2 The full model13.5 Training and validating the model13.5.1 The computeBatchLoss function13.5.2 The validation loop is similar13.6 Outputting performance metrics13.6.1 The logMetrics function13.7 Running the training script13.7.1 Data needed for training13.7.2 Interlude: The tqdm function13.8 Evaluating the model: Getting 99.7% correct means we’re done, right?13.9 Graphing training metrics with TensorBoard13.9.1 Running TensorBoard13.9.2 Adding TensorBoard support to the metrics logging function13.10 Why isn’t the model learning to detect nodules?13.11 Conclusion13.12 Exercises
14 Improving training with metrics and augmentation
14.1 High-level plan for improvement14.2 Good dogs vs. bad guys: False positives and false negatives14.3 Graphing the positives and negatives14.3.1 Recall is Chirpy’s strength14.3.2 Precision is Dozer’s forte14.3.3 Implementing precision and recall in logMetrics14.3.4 Our ultimate performance metric: The F1 score14.3.5 How does our model perform with our new metrics?14.4 What does an ideal dataset look like?14.4.1 Making the data look less like the actual and more like the “ideal”14.4.2 Contrasting training with a balanced LunaDataset to previous runs14.4.3 Recognizing the symptoms of overfitting14.5 Revisiting the problem of overfitting14.5.1 An overfit face-to-age prediction model14.6 Preventing overfitting with data augmentation14.6.1 Specific data augmentation techniques14.6.2 Seeing the improvement from data augmentation14.7 Conclusion14.8 Exercises
15 Using segmentation to find suspected nodules
15.1 Utilizing a second model in our project15.2 Various types of segmentation15.3 Semantic segmentation: Per-pixel classification15.3.1 The Segment Anything model (SAM)15.4 SAM architecture15.4.1 Trying out an off-the-shelf model for our project15.5 Using the SAM model directly15.6 Updating the dataset for segmentation15.6.1 Working around SAM’s limitation on 2D data15.6.2 Building the segmentation dataset15.6.3 Training a model to flag potential candidates15.7 Updating our training for fine-tuning15.7.1 How to fine-tune a model15.7.2 Using the AdamW optimizer15.7.3 Designing our training loop15.7.4 Saving our model15.8 Inference and results15.9 Conclusion15.10 Exercises
16 Training models on multiple GPUs
16.1 Introduction to parallel programming16.1.1 Distributed computing terminology16.1.2 Hardware requirements16.1.3 Initializing a distributed program16.2 Collective communication16.3 Introduction to parallelisms16.4 Data parallelism16.5 Model parallelism16.5.1 Pipeline parallelism16.5.2 Tensor parallelism16.5.3 Deciding between pipeline and tensor parallelism16.6 n-dimensional parallelism16.7 Fully sharded data parallelism16.8 Large language model–specific parallelisms16.8.1 Context parallelism16.8.2 Expert Parallelism16.9 Tying all parallelisms together16.10 Conclusion16.11 Exercises
17 Deploying to production
17.1 Serving PyTorch models17.1.1 Our model served by Gradio17.1.2 Our model behind a FastAPI server17.1.3 What we want from deployment17.1.4 Request batching and streaming responses17.1.5 How to make PyTorch models even faster17.2 Exporting models17.2.1 Interoperability beyond PyTorch with ONNX17.2.2 PyTorch’s own export: torch.export17.3 Expanding on torch.compile17.3.1 Full graph capture vs. disjoint graphs17.4 Understanding execution with torch.profiler17.5 Using PyTorch outside of Python17.5.1 LibTorch: PyTorch in C++17.6 Going mobile: ExecuTorch17.7 Conclusion17.8 Exercises

Content preview from Deep Learning with PyTorch, Second Edition

15 Using segmentation to find suspected nodules

This chapter covers

Modifying the data to be used for a 2D segmentation problem
Performing segmentation with Segment Anything
Understanding mask prediction using Segformer
Fine-tuning a segmentation model

In the last four chapters, we have accomplished a lot. We’ve learned about CT scans and lung tumors, datasets and data loaders, and metrics and monitoring. We have also applied many of the things we learned in part 1, and we have a working classifier that determines whether CT scan candidates are nodules or not nodules.

However, we are still operating in a somewhat artificial environment since we require hand-annotated nodule candidate information (our annotations.csv and candidates.csv ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781633438859Publisher Support Publisher Website Purchase Link

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Deep Learning with PyTorch, Second Edition

by Thomas Viehmann, Eli Stevens, Luca Pietro Giovanni Antiga, Howard Huang