Markov Chains: Analytic and Monte Carlo Computations

Table of Contents

1. Cover
2. Wiley Series in Probability and Statistics
3. Title Page
4. Copyright
5. Preface
6. List of Figures
7. Nomenclature
   1. Acronyms
   2. Symbols
8. Introduction
9. Chapter 1: First steps
   1. 1.1 Preliminaries
   2. 1.2 First properties of Markov chains
   3. 1.3 Natural duality: algebraic approach
   4. 1.4 Detailed examples
   5. Exercises
10. Chapter 2: Past, present, and future
    1. 2.1 Markov property and its extensions
    2. 2.2 Hitting times and distribution
    3. 2.3 Detailed examples
    4. Exercises
11. Chapter 3: Transience and recurrence
    1. 3.1 Sample paths and state space
    2. 3.2 Invariant measures and recurrence
    3. 3.3 Complements
    4. Exercises
12. Chapter 4: Long-time behavior
    1. 4.1 Path regeneration and convergence
    2. 4.2 Long-time behavior of the instantaneous laws
    3. 4.3 Elements on the rate of convergence for laws
    4. Exercises
13. Chapter 5: Monte Carlo methods
    1. 5.1 Approximate solution of the Dirichlet problem
    2. 5.2 Invariant law simulation
14. Appendix A: Complements
    1. A.1 Basic probabilistic notions
    2. A.2 Discrete measure convergence
    3. A.3 Measure-theoretic framework
15. References
16. Solutions for the exercises
    1. Solutions for Chapter 1
    2. Solutions for Chapter 2
17. Solutions for the exercises
    1. Solutions for Chapter 3
    2. Solutions for Chapter 4
18. Index
19. Wiley Series in Probability and Statistics
20. End User License Agreement