The state-value function measures the expected worth of each state in the process. As was suggested, the value can be described as the expected discounted total reward. Here is the mathematical formulation of the state value from the given state, s, at step t:
is a policy function that defines which action is taken by the agent. While our goal is to find a good policy function, the state-value function is fixed by the policy function. Thus, the state value is a scalar number that is a sum of discounted reward from the given ...
