Hands-On Intelligent Agents with OpenAI Gym

n-step advantage actor-critic algorithm

In the advantage actor-critic algorithm section, we looked at the steps involved in implementing the algorithm. In step 3, we noticed that we have to calculate the TD error based on the 1-step return (TD target). It is like letting the agent take a step in the environment and then based on the outcome, calculating the error in the critic's estimates and updating the policy of the agent. This sounds straightforward and simple, right? But, is there any better way to learn and update the policy? As you might have guessed from the title of this section, the idea is to use the n-step return, which uses more information to learn and update the policy compared to 1-step return-based TD learning. n-step TD ...