Markov Processes for Stochastic Modeling, 2nd Edition

14

Hidden Markov Models

14.1 Introduction

With the exception of partially observable Markov processes, all the Markov models we have considered up until now have visible states in the sense that the state sequence of the processes is known. Thus, we can refer to these models as visible Markov models. In this chapter, we consider a process in which the state sequence that the process passes through is not known but can only be guessed through a sequence of observations of the dynamics of the process. In the previous chapter, we considered a slightly different aspect of this model that we defined as the partially observable Markov decision process (POMDP). We devote this chapter to discussing another aspect of the model, which is called the hidden ...