On the Convergence Properties of the EM Algorithm

TLDR

The study investigates whether the EM algorithm converges to a local maximum or stationary point of the incomplete‑data likelihood and whether its sequence of parameter estimates converges. The authors enumerate key properties of the EM algorithm. The authors show that when the complete‑data model is a curved exponential family with compact parameter space, all EM limit points are stationary, and when the likelihood is unimodal with a suitable differentiability condition, any EM sequence converges to the unique maximum likelihood estimate.

Abstract

Two convergence aspects of the EM algorithm are studied: (i) does the EM algorithm find a local maximum or a stationary value of the (incomplete-data) likelihood function? (ii) does the sequence of parameter estimates generated by EM converge? Several convergence results are obtained under conditions that are applicable to many practical situations. Two useful special cases are: (a) if the unobserved complete-data specification can be described by a curved exponential family with compact parameter space, all the limit points of any EM sequence are stationary points of the likelihood function; (b) if the likelihood function is unimodal and a certain differentiability condition is satisfied, then any EM sequence converges to the unique maximum likelihood estimate. A list of key properties of the algorithm is included.