IEEE Signal Processing Letters · 2016 · 51 citations · 28 references
Natural SciencesMathematicsMachine LearningProbability TheoryStochastic ModelingEntropyMonte Carlo MethodsDivergenceDivergence MinimizationGaussian Mixture ModelsGuaranteed Upper BoundsKullback-leibler DivergenceMonte Carlo IntegrationUnivariate AnalysisUpper Bounds
The Kullback-Leibler (KL) divergence between two mixture models is a fundamental primitive in many signal processing tasks. Since the KL divergence of mixtures does not admit a closed-form formula, it is in practice either estimated using costly Monte-Carlo stochastic integration or approximated. We present a fast and generic method that builds algorithmically closed-form lower and upper bounds on the entropy, the cross-entropy and the KL divergence of univariate mixtures. We illustrate the versatile method by reporting on our experiments for approximating the KL divergence between Gaussian mixture models.
28
Clustering with Bregman Divergences
Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon et al. · 2004 · 1.5K citations
On entropy approximation for Gaussian mixture random vectors
Marco F. Huber, Tim Bailey, Hugh Durrant‐Whyte et al. · 2008 · 242 citations · Full text