Reinforcement Learning Algorithms with Python

Intuition behind NPG

Before looking at a potential solution to the instability of PG methods, let's understand why it appears. Imagine you are climbing a steep volcano with a crater on the top, similar to the function in the following diagram. Let's also imagine that the only sense you have is the inclination of your foot (the gradient) and that you cannot see the world around you—you are blind. Let's also set a fixed length of each step you can take (a learning rate), for example, one meter. You take the first step, perceive the inclination of your feet, and move 1 m toward the steepest ascent direction. After repeating this process many times, you arrive at a point near the top where the crater lies, but still, you are not aware of it since ...