Hands-On GPU Programming with Python and CUDA

Multi-GPU systems

The first major topic that comes to mind would be to learn how to program systems with more than one GPU installed. Many professional workstations and servers contain several GPUs that have been installed with the intention of processing far more data that requires not one, but several top-of-the-line GPUs. To this end, there exists a subfield called Multi-GPU programming. Much of the work is focused on load balancing, which is the art of using each GPU at its peak capacity, ensuring that no GPU gets saturated with too much work while the other goes without being fully utilized. Another topic here is Inter-GPU Communication, which is generally concerned about the issue of one GPU directly copying memory arrays to or from ...