Powerful Command-Line Applications in Go

Reduce Scheduling Contention

Goroutines are relatively cheap, and in some cases it makes sense to create many of them. For example, if the goroutines wait a long time on IO or network response, then the scheduler can execute other goroutines while these are waiting, increasing the efficiency of the program. In our case, the goroutines are mostly CPU-bound, so creating many of them doesn’t improve the efficiency. Instead, creating too many causes scheduling contention. Let’s address this issue by modifying our program to use worker queues. Instead of creating one goroutine per file, you’ll create one goroutine per available CPU. These will be our workers. Another goroutine sends the jobs to be executed by the workers. When no more jobs exist, ...