## Problems

17.1 Let

$\mu :=\mathbb{E}\left[f\left(\mathbf{x}\right)\right]=\int f\left(\mathbit{x}\right)p\left(\mathbit{x}\right)\text{d}\mathbit{x}$

and q(x) be the proposal distribution. Show that if

$w\left(\mathbit{x}\right):=\frac{p\left(\mathbit{x}\right)}{q\left(\mathbit{x}\right)},$

and

$\stackrel{^}{\mu }=\frac{1}{N}\sum _{i=1}^{N}w\left({\mathbit{x}}_{i}\right)f\left({\mathbit{x}}_{i}\right),$

then the variance

${\sigma }_{f}^{2}=\mathbb{E}\left[{\left(\stackrel{^}{\mathrm{\mu }}-\mathbb{E}\left[\stackrel{^}{\mathrm{\mu }}\right]\right)}^{2}\right]=\frac{1}{N}\left(\int \frac{{f}^{2}\left(\mathbit{x}\right){p}^{2}\left(\mathbit{x}\right)}{q\left(\mathbit{x}\right)}d\mathbit{x}-{\mu }^{2}\right).$

Observe that if f2(x)p2(x) goes to zero slower than q(x), then for fixed N, $σf2→∞$.

17.2 In importance sampling, with weights defined as

$w\left(\mathbit{x}\right)=\frac{\varphi \left(}{}$

Get Machine Learning now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.