Entropy · 2020 · 31 citations · 31 references
ComputingComputer ScienceComputer VisionInformation TheoryStatistical ComputingDimensionality ReductionProbability DistributionsGaussian Mixture ModelsHierarchical ClusteringMixture DistributionsMultivariate DistributionsMultivariate Normal DistributionRiemannian MetricShannon EntropySimilarity Measures
The Fisher-Rao distance is a measure of dissimilarity between probability distributions, which, under certain regularity conditions of the statistical model, is up to a scaling factor the unique Riemannian metric invariant under Markov morphisms. It is related to the Shannon entropy and has been used to enlarge the perspective of analysis in a wide variety of domains such as image processing, radar systems, and morphological classification. Here, we approach this metric considered in the statistical model of normal multivariate probability distributions, for which there is not an explicit expression in general, by gathering known results (closed forms for submanifolds and bounds) and derive expressions for the distance between distributions with the same covariance matrix and between distributions with mirrored covariance matrices. An application of the Fisher-Rao distance to the simplification of Gaussian mixtures using the hierarchical clustering algorithm is also presented.
31
On Information and Sufficiency
S. Kullback, R. A. Leibler · The Annals of Mathematical Statistics · 1951 · 20K citations · Full text