The Annals of Probability · 1996 · 281 citations · 6 references
Natural SciencesMathematicsInformation TheoryProbability TheoryStatisticsEntropyInformation MeasuresMeasure-theoretic ProbabilityProbability DistributionsProbability InequalitiesAlphabetsConcentration InequalitiesConcentration Of MeasureInformation Divergence
There is a simple inequality by Pinsker between variational distance and informational divergence of probability measures defined on arbitrary probability spaces. We shall consider probability measures on sequences taken from countable alphabets, and derive, from Pinsker's inequality, bounds on the $\bar{d}$-distance by informational divergence. Such bounds can be used to prove the "concentration of measure" phenomenon for some nonproduct distributions.
6
The positive-divergence and blowing-up properties
K. Marton, Paul C. Shields · Israel Journal of Mathematics · 1994 · 50 citations