Hands-On GPU Computing with Python

Implementing multiple GPUs with CuPy

When using more than one GPU on the same system, CuPy can be used to specify the exact GPU device to be worked with. By default, Device(0) is the first GPU.

To specify a second GPU, we can use the following, where the second GPU, Device(1), is being used to initialize p array:

cp.cuda.Device(1).use()p = np.zeros(N, dtype=np.double)

By default, all operations are done on Device(0), the first GPU. An array created on Device(1) will not work when used with Device(0).

You can also move an array from GPU 1 to GPU 2 just as from a CPU to a GPU, like so:

with cp.cuda.Device(0):gpu1 = np.zeros(N, dtype=np.double)with cp.cuda.Device(1):gpu2 = cp.asarray(gpu1) # Moving array to the second GPU

To confirm the GPU ...