Hands-On GPU Programming with Python and CUDA

Using shared memory

We can see from the prior example that the threads in the kernel can intercommunicate using arrays within the GPU's global memory; while it is possible to use global memory for most operations, we can speed things up by using shared memory. This is a type of memory meant specifically for intercommunication of threads within a single CUDA block; the advantage of using this over global memory is that it is much faster for pure inter-thread communication. In contrast to global memory, though, memory stored in shared memory cannot directly be accessed by the host—shared memory must be copied back into global memory by the kernel itself first.

Let's first step back for a moment before we continue and think about what we mean. ...