To improve on robotic movement, OpenAI researchers released a paper on a technique known as Hindsight experience replay (HER) in 2018 to attempt to overcome issues that arise when training reinforcement learning algorithms in sparse environments. Recall Chapter 8, Reinforcement Learning, choosing an appropriate reward for an agent can make or break the performance of that agent. Anecdotes such as the following often appear as the result of bad reward functions:
A friend is training a simulated robot arm to reach toward a point above a table. It turns out the point was defined with respect to the table, and the table wasn’t anchored to anything. The policy learned to slam the table really hard, making the table ...
