Entropy · 2011 · 128 citations · 6 references
Natural SciencesMathematicsInformation TheoryProbability TheorySet TheoryEntropyInformation MeasuresLoss FunctionRandom VariablesConditional EntropyInformation EntropyInformation Loss AnalysisShannon EntropyTsallis Entropy
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the “information loss”, or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well.
6
A Mathematical Theory of Communication
Claude E. Shannon · Bell System Technical Journal · 1948 · 80.7K citations
Diversity as a Concept and its Measurement
G. P. Patil, C. Taillie · Journal of the American Statistical Association · 1982 · 725 citations
Life Sciences, Biology, Ecology +14
Diversity as a Concept and its Measurement
G. P. Patil, C. Taillie · Journal of the American Statistical Association · 1982 · 158 citations
Life Sciences, Biology, Ecology +14