Hierarchical Mixture Modeling With Normalized Inverse-Gaussian Priors

TLDR

The Dirichlet process prior has become popular in Bayesian mixture modeling because hierarchical models and efficient sampling techniques can overcome its inherent discreteness. This study proposes the normalized inverse‑Gaussian process as an alternative to the Dirichlet process for use in Bayesian hierarchical models. The N‑IG prior is defined through its finite‑dimensional distributions, and its mixture is compared to the Dirichlet mixture on two normal‑mixture examples, showing that existing sampling schemes can be readily adapted. The N‑IG prior retains discreteness but yields richer clustering by weighting observations according to the total number of ties, while its moments and predictive distributions remain tractable, enabling straightforward extension of established sampling methods.

Abstract

In recent years the Dirichlet process prior has experienced a great success in the context of Bayesian mixture modeling. The idea of overcoming discreteness of its realizations by exploiting it in hierarchical models, combined with the development of suitable sampling techniques, represent one of the reasons of its popularity. In this article we propose the normalized inverse-Gaussian (N–IG) process as an alternative to the Dirichlet process to be used in Bayesian hierarchical models. The N–IG prior is constructed via its finite-dimensional distributions. This prior, although sharing the discreteness property of the Dirichlet prior, is characterized by a more elaborate and sensible clustering which makes use of all the information contained in the data. Whereas in the Dirichlet case the mass assigned to each observation depends solely on the number of times that it occurred, for the N–IG prior the weight of a single observation depends heavily on the whole number of ties in the sample. Moreover, expressions corresponding to relevant statistical quantities, such as a priori moments and the predictive distributions, are as tractable as those arising from the Dirichlet process. This implies that well-established sampling schemes can be easily extended to cover hierarchical models based on the N–IG process. The mixture of N–IG process and the mixture of Dirichlet process are compared using two examples involving mixtures of normals.