Digital Communication Systems

APPENDIX

I   Monte Carlo Integration

In a generic sense, Monte Carlo simulation¹ is an invaluable experimental tool for tackling difficult problems that are mathematically intractable; but the tool is imprecise in that it provides statistical estimates. Nevertheless, provided that the Monte Carlo simulation is conducted properly, valuable insight into a problem of interest is obtained, which would be difficult otherwise.

In this appendix, we focus on Monte Carlo integration, which is a special form of Monte Carlo simulation. Specifically, we address the difficult integration problem encountered in Chapter 5 dealing with computation of the differential entropy h(Y), based on the mathematically intractable conditional probability density function of (5.102) in Chapter 5.

To elaborate, we may say:

Monte Carlo integration is a computational tool, which is used to integrate a given function defined over a prescribed area of interest that is not easy to sample in a random and uniform manner.

Let W denote the difficult area over which random sampling of the differential entropy h(Y) is to be performed. To get around this difficulty, let V denote an area so configured that it incudes the area W and is easy to randomly sample. Desirably, the selected area V enclosed W as closely as possible for the simple reason that samples picked outside of W are of no practical interest.

Suppose now we pick a total of N samples in the area V, randomly and uniformly. Then according to Press, et al. (1998), ...