Efficient C++ Performance Programming Techniques

Thundering Herd

When a thread fails to acquire a busy lock, it can proceed in one of two ways:

• Spin in a tight loop trying to grab the lock on each iteration. This is effective only for short-lived locks. Spin-locks that are held for long periods of time will cause threads to tie up the CPUs while spinning idle waiting on the shared resource to become available.

• Go to sleep and await notification. When the lock is freed, waiting threads are awakened. We'll call this type a simple-lock to distinguish it from the spin-lock.

When a large number of threads concurrently contend for a simple-lock, all but one are put to sleep. What happens when the lock finally becomes available? Again, there are two scenarios:

• Only a single thread on the waiting list ...