Hands-On Machine Learning for Algorithmic Trading

Value functions – how to estimate the long-run reward

As we mentioned previously, a policy, π, maps all states to probability distributions over actions so that the probability of choosing action A_t in state S_t can be expressed as . The value function estimates the long-run return for each state or state-action pair. It is fundamental to find the policy that is the optimal mapping of states to actions.

The state-value function for the π policy gives the long-term value, v, of a state, S as the expected return, G, for an agent that starts in ...