Nonlinear Filters

10Expectation Maximization

10.1 Introduction

In machine learning and statistics, computing the maximum likelihood (ML) or maximum a posteriori (MAP) parameter estimate relies on the availability of complete data. However, if the model consists of latent variables or there is missing data, then ML and MAP estimation will become a challenging problem. In such cases, gradient descent methods can be used to find a local minimum of the negative log likelihood (NLL) [77]:

(10.1)

where represents the set of model parameters and denotes the set of observed data points for . It is often required to impose additional constraints on the model such as mixing weights must be normalized and covariance matrices must be positive definite. Expectation maximization (EM) algorithm paves the way for addressing this issue. As an iterative algorithm, EM enforces the required constraints and handles the missing data problem by alternating between two steps [77]:

• E‐step: inferring the missing values given ...