A First Course in Probability and Markov Chains

A First Course in Probability and Markov Chains

by Giuseppe Modica, Laura Poggiolini
Released January 2013
Publisher(s): Wiley
ISBN: 9781119944874

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Provides an introduction to basic structures of probability with a view towards applications in information technology

A First Course in Probability and Markov Chains presents an introduction to the basic elements in probability and focuses on two main areas. The first part explores notions and structures in probability, including combinatorics, probability measures, probability distributions, conditional probability, inclusion-exclusion formulas, random variables, dispersion indexes, independent random variables as well as weak and strong laws of large numbers and central limit theorem. In the second part of the book, focus is given to Discrete Time Discrete Markov Chains which is addressed together with an introduction to Poisson processes and Continuous Time Discrete Markov Chains. This book also looks at making use of measure theory notations that unify all the presentation, in particular avoiding the separate treatment of continuous and discrete distributions.

A First Course in Probability and Markov Chains:

  • Presents the basic elements of probability.

  • Explores elementary probability with combinatorics, uniform probability, the inclusion-exclusion principle, independence and convergence of random variables.

  • Features applications of Law of Large Numbers.

  • Introduces Bernoulli and Poisson processes as well as discrete and continuous time Markov Chains with discrete states.

  • Includes illustrations and examples throughout, along with solutions to problems featured in this book.

The authors present a unified and comprehensive overview of probability and Markov Chains aimed at educating engineers working with probability and statistics as well as advanced undergraduate students in sciences and engineering with a basic background in mathematical analysis and linear algebra.

Table of contents

  1. Coverpage
  2. Titlepage
  3. Copyright
  4. Contents
  5. Preface
  6. 1 Combinatorics
    1. 1.1 Binomial coefficients
      1. 1.1.1 Pascal triangle
      2. 1.1.2 Some properties of binomial coefficients
      3. 1.1.3 Generalized binomial coefficients and binomial series
      4. 1.1.4 Inversion formulas
      5. 1.1.5 Exercises
    2. 1.2 Sets, permutations and functions
      1. 1.2.1 Sets
      2. 1.2.2 Permutations
      3. 1.2.3 Multisets
      4. 1.2.4 Lists and functions
      5. 1.2.5 Injective functions
      6. 1.2.6 Monotone increasing functions
      7. 1.2.7 Monotone nondecreasing functions
      8. 1.2.8 Surjective functions
      9. 1.2.9 Exercises
    3. 1.3 Drawings
      1. 1.3.1 Ordered drawings
      2. 1.3.2 Simple drawings
      3. 1.3.3 Multiplicative property of drawings
      4. 1.3.4 Exercises
    4. 1.4 Grouping
      1. 1.4.1 Collocations of pairwise different objects
      2. 1.4.2 Collocations of identical objects
      3. 1.4.3 Multiplicative property
      4. 1.4.4 Collocations in statistical physics
      5. 1.4.5 Exercises
  7. 2 Probability measures
    1. 2.1 Elementary probability
      1. 2.1.1 Exercises
    2. 2.2 Basic facts
      1. 2.2.1 Events
      2. 2.2.2 Probability measures
      3. 2.2.3 Continuity of measures
      4. 2.2.4 Integral with respect to a measure
      5. 2.2.5 Probabilities on finite and denumerable sets
      6. 2.2.6 Probabilities on denumerable sets
      7. 2.2.7 Probabilities on uncountable sets
      8. 2.2.8 Exercises
    3. 2.3 Conditional probability
      1. 2.3.1 Definition
      2. 2.3.2 Bayes formula
      3. 2.3.3 Exercises
    4. 2.4 Inclusion–exclusion principle
      1. 2.4.1 Exercises
  8. 3 Random variables
    1. 3.1 Random variables
      1. 3.1.1 Definitions
      2. 3.1.2 Expected value
      3. 3.1.3 Functions of random variables
      4. 3.1.4 Cavalieri formula
      5. 3.1.5 Variance
      6. 3.1.6 Markov and Chebyshev inequalities
      7. 3.1.7 Variational characterization of the median and of the expected value
      8. 3.1.8 Exercises
    2. 3.2 A few discrete distributions
      1. 3.2.1 Bernoulli distribution
      2. 3.2.2 Binomial distribution
      3. 3.2.3 Hypergeometric distribution
      4. 3.2.4 Negative binomial distribution
      5. 3.2.5 Poisson distribution
      6. 3.2.6 Geometric distribution
      7. 3.2.7 Exercises
    3. 3.3 Some absolutely continuous distributions
      1. 3.3.1 Uniform distribution
      2. 3.3.2 Normal distribution
      3. 3.3.3 Exponential distribution
      4. 3.3.4 Gamma distributions
      5. 3.3.5 Failure rate
      6. 3.3.6 Exercises
  9. 4 Vector valued random variables
    1. 4.1 Joint distribution
      1. 4.1.1 Joint and marginal distributions
      2. 4.1.2 Exercises
    2. 4.2 Covariance
      1. 4.2.1 Random variables with finite expected value and variance
      2. 4.2.2 Correlation coefficient
      3. 4.2.3 Exercises
    3. 4.3 Independent random variables
      1. 4.3.1 Independent events
      2. 4.3.2 Independent random variables
      3. 4.3.3 Independence of many random variables
      4. 4.3.4 Sum of independent random variables
      5. 4.3.5 Exercises
    4. 4.4 Sequences of independent random variables
      1. 4.4.1 Weak law of large numbers
      2. 4.4.2 Borel–Cantelli lemma
      3. 4.4.3 Convergences of random variables
      4. 4.4.4 Strong law of large numbers
      5. 4.4.5 A few applications of the law of large numbers
      6. 4.4.6 Central limit theorem
      7. 4.4.7 Exercises
  10. 5 Discrete time Markov chains
    1. 5.1 Stochastic matrices
      1. 5.1.1 Definitions
      2. 5.1.2 Oriented graphs
      3. 5.1.3 Exercises
    2. 5.2 Markov chains
      1. 5.2.1 Stochastic processes
      2. 5.2.2 Transition matrices
      3. 5.2.3 Homogeneous processes
      4. 5.2.4 Markov chains
      5. 5.2.5 Canonical Markov chains
      6. 5.2.6 Exercises
    3. 5.3 Some characteristic parameters
      1. 5.3.1 Steps for a first visit
      2. 5.3.2 Probability of (at least) r visits
      3. 5.3.3 Recurrent and transient states
      4. 5.3.4 Mean first passage time
      5. 5.3.5 Hitting time and hitting probabilities
      6. 5.3.6 Exercises
    4. 5.4 Finite stochastic matrices
      1. 5.4.1 Canonical representation
      2. 5.4.2 States classification
      3. 5.4.3 Exercises
    5. 5.5 Regular stochastic matrices
      1. 5.5.1 Iterated maps
      2. 5.5.2 Existence of fixed points
      3. 5.5.3 Regular stochastic matrices
      4. 5.5.4 Characteristic parameters
      5. 5.5.5 Exercises
    6. 5.6 Ergodic property
      1. 5.6.1 Number of steps between consecutive visits
      2. 5.6.2 Ergodic theorem
      3. 5.6.3 Powers of irreducible stochastic matrices
      4. 5.6.4 Markov chain Monte Carlo
    7. 5.7 Renewal theorem
      1. 5.7.1 Periodicity
      2. 5.7.2 Renewal theorem
      3. 5.7.3 Exercises
  11. 6 An introduction to continuous time Markov chains
    1. 6.1 Poisson process
    2. 6.2 Continuous time Markov chains
      1. 6.2.1 Definitions
      2. 6.2.2 Continuous semigroups of stochastic matrices
      3. 6.2.3 Examples of right-continuous Markov chains
      4. 6.2.4 Holding times
  12. Appendix A Power series
    1. A.1 Basic properties
    2. A.2 Product of series
    3. A.3 Banach space valued power series
      1. A.3.2 Exercises
  13. Appendix B Measure and integration
    1. B.1 Measures
      1. B.1.1 Basic properties
      2. B.1.2 Construction of measures
      3. B.1.3 Exercises
    2. B.2 Measurable functions and integration
      1. B.2.1 Measurable functions
      2. B.2.2 The integral
      3. B.2.3 Properties of the integral
      4. B.2.4 Cavalieri formula
      5. B.2.5 Markov inequality
      6. B.2.6 Null sets and the integral
      7. B.2.7 Push forward of a measure
      8. B.2.8 Exercises
    3. B.3 Product measures and iterated integrals
      1. B.3.1 Product measures
      2. B.3.2 Reduction formulas
      3. B.3.3 Exercises
    4. B.4 Convergence theorems
      1. B.4.1 Almost everywhere convergence
      2. B.4.2 Strong convergence
      3. B.4.3 Fatou lemma
      4. B.4.4 Dominated convergence theorem
      5. B.4.5 Absolute continuity of integrals
      6. B.4.6 Differentiation of the integral
      7. B.4.7 Weak convergence of measures
      8. B.4.8 Exercises
  14. Appendix C Systems of linear ordinary differential equations
    1. C.1 Cauchy problem
      1. C.1.1 Uniqueness
      2. C.1.2 Existence
    2. C.2 Efficient computation of eQt
      1. C.2.1 Similarity methods
      2. C.2.2 Putzer method
    3. C.3 Continuous semigroups
  15. References
  16. Index

Product information

  • Title: A First Course in Probability and Markov Chains
  • Author(s): Giuseppe Modica, Laura Poggiolini
  • Release date: January 2013
  • Publisher(s): Wiley
  • ISBN: 9781119944874