Convergence of the EM Algorithm for Gaussian Mixtures with Unbalanced\n Mixing Coefficients

Abstract

The speed of convergence of the Expectation Maximization (EM) algorithm for\nGaussian mixture model fitting is known to be dependent on the amount of\noverlap among the mixture components. In this paper, we study the impact of\nmixing coefficients on the convergence of EM. We show that when the mixture\ncomponents exhibit some overlap, the convergence of EM becomes slower as the\ndynamic range among the mixing coefficients increases. We propose a\ndeterministic anti-annealing algorithm, that significantly improves the speed\nof convergence of EM for such mixtures with unbalanced mixing coefficients. The\nproposed algorithm is compared against other standard optimization techniques\nlike BFGS, Conjugate Gradient, and the traditional EM algorithm. Finally, we\npropose a similar deterministic anti-annealing based algorithm for the\nDirichlet process mixture model and demonstrate its advantages over the\nconventional variational Bayesian approach.\n