Hands-On Reinforcement Learning for Games

Monte Carlo control

There are two ways to implement what is called Monte Carlo control on an agent. The difference between the two is how they calculate the average return or sampled mean. In what is called First-Visit Monte Carlo, the agent only samples the mean the first time a state is visited. The other method, Every-Visit Monte Carlo, samples the average return every time a state is visited. The latter method is what we will explore in the code example for this chapter.

The original source code for this example was from Ankit Choudhary's blog (https://www.analyticsvidhya.com/blog/2018/11/reinforcement-learning-introduction-monte-carlo-learning-openai-gym/).

The code has been heavily modified from the original. Ankit goes far more heavily ...