Hands-On Intelligent Agents with OpenAI Gym

Episodic life

In many games, including Atari games, the player gets to play with more than one life.

It was observed, used, and reported by Deepmind that terminating an episode when a life is lost helps the agent learn better. It has to be noted that the intention is to signify to the agent that losing a life is a bad thing to do. In this case, when the episode terminates, we will not reset the environment and rather continue until the game is actually over, after which we reset the environment. If we reset the game after every loss of life, we would be limiting the agent's exposure to observations and experiences that can be collected with just one life, which is usually bad for the agent's learning performance.

To implement what we just ...