Reinforcement Learning with TensorFlow

Actor-critic algorithms

The preceding policy optimization using the Monte Carlo policy gradient approach leads to high variance. In order to tackle this issue, we use a critic to estimate the state-action value function, that is as follows:

This gives rise to the famous actor-critic algorithms. The actor-critic algorithm, as the name suggests, maintains two networks for the following purposes:

• One network acts as a critic, which updates the weight w parameter vector of the function approximator of the state-action
• Other network acts as an Actor, which updates the policy parameter vector  as per the direction given by the critic

The following ...