Reinforcement Learning Algorithms with Python

Scalable implementation

To simplify the implementation and to make the parallelized version of ES work well with a limited number of workers (and CPUs), we will develop a structure similar to the one that's shown in the following diagram. The main process creates one worker for each CPU core and executes the main cycle. On each iteration, it waits until a given number of new candidates are evaluated by the workers. Different from the implementation provided in the paper, each worker evaluates more than one agent on each iteration. So, if we have four CPUs, four workers will be created. Then, if we want a total batch size bigger than the number of workers on each iteration of the main process, let's say, 40, each worker will create and evaluate ...