IEEE Transactions on Information Theory · 2010 · 40 citations · 25 references
Natural SciencesMathematicsDiscrete MathematicsProbability TheoryStochastic ProcessesEntropyInformation MeasuresProbability InequalitiesContinuous Random VariablesDiscrete Random VariablesDivisibility ProofsEntropy BoundsEntropy InequalityInfinite DivisibilityPower InequalityRényi Entropy
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for Poisson variables. We show that some natural analogues of the EPI do not in fact hold, but propose an alternative formulation which does always hold. The key to many proofs of Shannon's EPI is the behavior of entropy on scaling of continuous random variables. We believe that Rényi's operation of thinning discrete random variables plays a similar role to scaling, and give a sharp bound on how the entropy of ultra log-concave random variables behaves on thinning. In the spirit of the monotonicity results established by Artstein, Ball, Barthe, and Naor, we prove a stronger version of concavity of entropy, which implies a strengthened form of our discrete EPI.
25
A Mathematical Theory of Communication
Claude E. Shannon · Bell System Technical Journal · 1948 · 80.7K citations
<i>The Mathematical Theory of Communication</i>
Claude E. Shannon, Warren Weaver, Norbert Wiener · Physics Today · 1950 · 9.9K citations
Inequalities in Fourier Analysis
William Beckner · Annals of Mathematics · 1975 · 1.3K citations
Information theoretic inequalities
Amir Dembo, Thomas M. Cover, J. Thomas · IEEE Transactions on Information Theory · 1991 · 757 citations