Python Deep Learning - Second Edition

Policy evaluation

In the Dynamic programming section, we'll describe how to estimate the value function, , given a policy, π (planning). MC does this by playing full episodes, and then averaging the cumulative returns for each state over the different episodes.

Let's see how it works in the following steps:

1. Input the policy, π.
2. Initialize the following:
   • Thetable with some value for all states
   • An empty list of returns(s) for each state, s
3. For a number of episodes, do the following:
   1. Generate a new episode, following policy π: s₀, a₀, r₁, s ...