Variational learning for Gaussian mixture models

TLDR

In Bayesian inference, parameter distributions are modeled with hyperparameters. The paper proposes a joint maximum‑likelihood and Bayesian approach to estimate Gaussian mixture models and introduces a hyperparameter initialization procedure for training. The method models means with Gaussian, covariances with Wishart, and mixing weights with Dirichlet, estimates hyperparameters via variational expectation‑maximization, and initializes them by fitting EM runs to obtain maximum‑likelihood estimates, then applies the approach to blind signal detection and color image segmentation. The proposed initialization yields faster convergence, more accurate hyperparameter estimates, and better generalization for the VEM training algorithm.

Abstract

This paper proposes a joint maximum likelihood and Bayesian methodology for estimating Gaussian mixture models. In Bayesian inference, the distributions of parameters are modeled, characterized by hyperparameters. In the case of Gaussian mixtures, the distributions of parameters are considered as Gaussian for the mean, Wishart for the covariance, and Dirichlet for the mixing probability. The learning task consists of estimating the hyperparameters characterizing these distributions. The integration in the parameter space is decoupled using an unsupervised variational methodology entitled variational expectation-maximization (VEM). This paper introduces a hyperparameter initialization procedure for the training algorithm. In the first stage, distributions of parameters resulting from successive runs of the expectation-maximization algorithm are formed. Afterward, maximum-likelihood estimators are applied to find appropriate initial values for the hyperparameters. The proposed initialization provides faster convergence, more accurate hyperparameter estimates, and better generalization for the VEM training algorithm. The proposed methodology is applied in blind signal detection and in color image segmentation