Dealing With Label Switching in Mixture Models

TLDR

In Bayesian finite mixture models, parameter estimation and clustering can be misleading because label switching—arising from likelihood symmetry—renders posterior summaries nonsensical, and the usual remedy of imposing artificial identifiability constraints often fails. The study aims to show that artificial identifiability constraints do not generally resolve label switching and to propose relabelling algorithms based on minimizing posterior expected loss. The authors develop relabelling algorithms that minimize posterior expected loss under various loss functions, presenting a simple, general algorithm as a concrete example. The algorithm successfully mitigates label switching in two illustrative examples, demonstrating its practical effectiveness.

Abstract

Summary In a Bayesian analysis of finite mixture models, parameter estimation and clustering are sometimes less straightforward than might be expected. In particular, the common practice of estimating parameters by their posterior mean, and summarizing joint posterior distributions by marginal distributions, often leads to nonsensical answers. This is due to the so-called ‘label switching’ problem, which is caused by symmetry in the likelihood of the model parameters. A frequent response to this problem is to remove the symmetry by using artificial identifiability constraints. We demonstrate that this fails in general to solve the problem, and we describe an alternative class of approaches, relabelling algorithms, which arise from attempting to minimize the posterior expected loss under a class of loss functions. We describe in detail one particularly simple and general relabelling algorithm and illustrate its success in dealing with the label switching problem on two examples.