Hands-On Reinforcement Learning for Games

The TRPO step

As you can see now, taking a step or update with TRPO is not trivial and things are still going to get more complicated. The step itself requires the agent to learn several factors from updating the policy and value function to also attain an advantage, also known as actor-critic. Understanding the actual details of the step function is beyond the scope of this book and you are again referred to those external references. However, it may be helpful to review what constitutes a step in TRPO and how this may compare complexity wise to other methods we look at in the future. Open up the sample TRPO folder again and follow the next exercise:

1. Open up the main.py file and find the following line of code, around line 130:

trpo_step(policy_net, ...