Hands-On Reinforcement Learning for Games

Introducing DP

DP was developed by Richard E. Bellman in the 1950s as a way to optimize and solve complex decision problems. The method was first applied to engineering control problems but has since found uses in all disciplines requiring the analytical modeling of problems and subproblems. In effect, all DP is about is solving subproblems and then finding relationships to connect those to solve bigger problems. It does all of this by first applying the Bellman optimality equation and then solving it.

Before we get to solving a finite MDP with DP, we will want to understand, in a little more detail, what it is we are talking about. Let's look at a simple example of the difference between normal recursion and DP in the next section.