Actor-Critic TD(0) in the checkerboard environment

In this example, we want to employ an alternative algorithm called Actor-Critic, together with TD(0). In this method, the agent is split into two components, a Critic, which is responsible for evaluating the quality of the value estimation, and an actor, which selects and performs an action. As pointed out by Dayan (in Theoretical NeuroscienceDayan P., Abbott LF., The MIT Press), the dynamics of an Actor-Critic approach are similar to the interleaving policy evaluation and policy improvement steps. In fact, the knowledge of the Critic is obtained through an iterative process, and its initial evaluations are normally sub-optimal.

The structural schema is shown in the following diagram: ...

Get Mastering Machine Learning Algorithms now with the O’Reilly learning platform.

O’Reilly members experience live online training, plus books, videos, and digital content from nearly 200 publishers.