Skip to Main Content
Accelerate Model Training with PyTorch 2.X
book

Accelerate Model Training with PyTorch 2.X

by Maicon Melo Alves
April 2024
Intermediate to advanced content levelIntermediate to advanced
230 pages
5h 12m
English
Packt Publishing
Content preview from Accelerate Model Training with PyTorch 2.X

10

Training with Multiple GPUs

Undoubtedly, the computing power provided by GPUs is one of the factors that’s responsible for boosting the deep learning area. If a single GPU device can accelerate the training process exceedingly, imagine what we can do with a multi-GPU environment.

In this chapter, we will show you how to use multiple GPUs to accelerate the training process. Before describing the code and launching procedure, we will dive into the characteristics and nuances of the multi-GPU environment.

Here is what you will learn as part of this chapter:

  • The fundamentals of a multi-GPU environment
  • How to distribute the training process among multiple GPUs
  • NCCL, the default backend for distributed training on NVIDIA GPUs

Technical requirements ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

PyTorch Recipes: A Problem-Solution Approach to Build, Train and Deploy Neural Network Models

PyTorch Recipes: A Problem-Solution Approach to Build, Train and Deploy Neural Network Models

Pradeepta Mishra

Publisher Resources

ISBN: 9781805120100Supplemental Content