Machine Learning

12.5 Latent Variables and the EM Algorithm

At the end of Section 12.3, it was pointed out that the evidence function associated with the regression task in Eq. (12.3), assuming that p(y|θ) and p(θ) are Gaussians of the form given in (12.39), is also Gaussian parameterized via a set of parameters, ξ, where for this case $\mathit{\xi }=[{\sigma }_{\eta }^2,{\sigma }_{\theta }^2]$, and we can write p(y;ξ). Maximizing the evidence with respect to ξ becomes a typical ML task. However, in general, such closed-form expressions for the evidence function are not possible, and the integration in (12.14) is intractable. The main source of difficulty is the fact that our regression model is described ...