Deep Learning with PyTorch Quick Start Guide

torch.multiprocessing

The torch.multiprocessor package is a replacement for the Python multiprocessor package, and is used in exactly the same way, that is, as a process-based threading interface. One of the ways it extends the Python distributed package is by placing PyTorch tensors into shared memory and only sending their handles to other processes. This is achieved using a multiprocessing.Queue object. In general, multiprocessing occurs asynchronously; that is, processes for a particular device are enqueued and executed when the process reaches the top of the queue. Each device executes a process in the order that it is queued and PyTorch periodically synchronizes the multiple processes when copying between devices. This means that, as ...