Skip to Main Content
Bayesian Analysis of Stochastic Process Models
book

Bayesian Analysis of Stochastic Process Models

by David Insua, Fabrizio Ruggeri, Mike Wiper
May 2012
Intermediate to advanced content levelIntermediate to advanced
332 pages
8h 39m
English
Wiley
Content preview from Bayesian Analysis of Stochastic Process Models

3

Discrete time Markov chains and extensions

3.1 Introduction

As we mentioned in Chapter 1, Markov chains are one of the simplest stochastic processes to study and are characterized by a lack of memory property, so that future observations depend only on the current state and not on the whole of the past history of the process. Despite their simplicity, Markov chains can be and have been applied to many real problems in areas as diverse as web-browsing behavior, language modeling, and persistence of surnames over generations. Furthermore, as illustrated in Chapter 2, with the development of Markov chain Monte Carlo (MCMC) methods, Markov chains have become a basic tool for Bayesian analysis.

In this chapter, we shall study the Bayesian analysis of discrete time Markov chains, focusing on homogeneous chains with a finite state space. We shall also analyze many important subclasses and extensions of this basic model such as reversible chains, branching processes, higher order Markov chains, and discrete time Markov processes with continuous state spaces. The properties of the basic Markov chain model and these variants are outlined from a probabilistic viewpoint in Section 3.2.

In Section 3.3, inference for time homogeneous, discrete state space, first-order chains is considered. Then, Section 3.4 provides inference for various extensions and particular classes of chains. A case study on the analysis of wind directions is presented in Section 3.5 and Markov decision processes are ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Case Studies in Bayesian Statistical Modelling and Analysis

Case Studies in Bayesian Statistical Modelling and Analysis

Clair L. Alston, Kerrie L. Mengersen, Anthony N. Pettitt
Learning Bayesian Models with R

Learning Bayesian Models with R

Hari Manassery Koduvely
Applied Modeling Techniques and Data Analysis 2

Applied Modeling Techniques and Data Analysis 2

Yannis Dimotikalis, Alex Karagrigoriou, Christina Parpoula, Christos H. Skiadas

Publisher Resources

ISBN: 9781118304037Purchase book