Hands-On Reinforcement Learning with Python

State value function

A state value function is also called simply a value function. It specifies how good it is for an agent to be in a particular state with a policy π. A value function is often denoted by V(s). It denotes the value of a state following a policy.

We can define a state value function as follows:

This specifies the expected return starting from state s according to policy π. We can substitute the value of R_t in the value function from (2) as follows:

Note that the state value function depends on the policy and it varies depending ...