The goal of the agent is to maximize the total cumulative reward in the long run. The policy, which maximizes the total cumulative reward is called the optimal policy and is denoted with denoted with 
. There could be different optimal policies, but they all share the same value functions (optimal value functions).
We'll denote the state-value and action-value functions with respect to the optimal policy, 
, with the following:
Let's expand on the optimal action-value function. First, we start with a state, ...
