Reinforcement Learning with TensorFlow

State estimation

If we expand the state spaces, this helps us to convert the POMDP into an MDP where Z contains fully observable state space. This gives the notion of belief state b(s), which is the state that the decision maker is going to use in the context of a POMDP . The belief state, that is, b(s) gives the probability of the agent being in the s state. Therefore, belief state, b, is a vector representing the probability distribution over all states. Thus, the belief state gets updated as soon as an action is taken.

Say, there's a belief state, b, the agent takes an action, a, and received some observations, z. This forms a new belief state. Therefore, we are converting a POMDP to belief MDP where it will consist of belief states as ...