Expert Shell Scripting

CHAPTER 14Throttling Parallel Processes

I have often needed to perform a task across multiple remote systems. A common example is the installation of a software package on each machine in a large environment. With relatively small environments, you could simply write a script that loops through a list of systems and performs the desired task serially on each machine. Another method would be to loop through the list of machines and submit your job to the background so the tasks are performed in parallel. Neither of these methods scales well when you run a large environment, however. Processing the list sequentially is not an efficient use of resources and can take a long time to complete.

With too many background parallel processes, the initiating ...