Hands-On GPU Programming with Python and CUDA

Host-side multiprocessing and multithreading

Of course, we may seek to gain concurrency on the host side by using multiple processes or threads on the host's CPU. Let's make the distinction right now between a host-side operating system process and thread with a quick overview.

Every host-side program that exists outside the operating system kernel is executed as a process, and can also exist in multiple processes. A process has its own address space, as it runs concurrently with, and independently of, all other processes. A process is, generally speaking, blind to the actions of other processes, although multiple processes can communicate through sockets or pipes. In Linux and Unix, new processes are spawned with the fork system call.