Hands-On Q-Learning with Python

Introducing CartPole-v1

Your task in the CartPole environment is simple: move a cart back and forth along a wire so that a pole pivoting on the cart balances upright. In control theory, this is called the inverted pendulum problem, and it is one of several classic control theory problems implemented as reinforcement learning environments in OpenAI Gym.

Here's an illustration of the Gym implementation of the task:

The inverted pendulum as defined in control theory is an underactuated system, meaning it has more degrees of freedom than actuated (controllable) types of movement.

In other words, the position of the cart can be directly controlled, ...