PyTorch Recipes: A Problem-Solution Approach to Build, Train and Deploy Neural Network Models

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2023

P. MishraPyTorch Recipeshttps://doi.org/10.1007/978-1-4842-8925-9_8

8. Distributed PyTorch Modelling, Model Optimization, and Deployment

Pradeepta Mishra¹  

(1)

Bangalore, Karnataka, India

 

In this chapter, you will use PyTorch to implement the steps that are most commonly used in installation, training, and setting up distributed PyTorch for model training. The architecture followed for distributed data parallel training and distributed model parallel training can be explained using the following figures. The model optimization process reduces the model parameter’s size so that the model object becomes lighter. The bigger the model object, the slower the inference ...