Scaling Python with Ray

Chapter 11. Using GPUs and Accelerators with Ray

While Ray is primarily focused on horizontal scaling, sometimes using special accelerators like GPUs can be cheaper and faster than just throwing more “regular” compute nodes at a problem. GPUs are well suited to vectorized operations performing the same operation on chunks of data at a time. ML, and more generally linear algebra, are some of the top use cases,¹ as deep learning is incredibly vectorizable.

Often GPU resources are more expensive than CPU resources, so Ray’s architecture makes it easy to request GPU resources only when necessary. To take advantage of GPUs, you need to use specialized libraries, and since these libraries deal with direct memory access, their results may not always be serializable. In the GPU computing world, NVIDIA and, to a lesser degree, AMD are the two main options, with different libraries for integration.

What Are GPUs Good At?

Not every problem is a good fit for GPU acceleration. GPUs are especially good at performing the same calculation on many data points at the same time. If a problem is well suited to vectorization, there is a good chance that GPUs may be well suited to it.

The following are common problems that benefit from GPU acceleration:

• ML

• Linear algebra

• Physics simulations

• Graphics (no surprise here)

GPUs are not well suited to branch-heavy nonvectorized workflows, or workflows for which the cost of copying the data is similar to or higher than the cost of the computation. ...