Hands-On Reinforcement Learning for Games

Discretizing continuous state spaces

RL is limited to discrete spaces or what we learned previously as a finite MDP. A finite MDP describes a discrete set of steps or states with an action to move between states decided by a probability. The infinite version of this may define an infinite number of state-actions between any set of states. Hence, a basketball player moving from one end of the court to score a basket describes an infinite MDP or continuous space. That is, for each point in time, the ball player could be in an infinite number of positions, dribbling or not dribbling, or shooting the ball and so on. Likewise, in the MountainCar environment, the car can be moving up or down the hill in either direction at any point in time. This ...