Concepedia

Information Divergence

Information Divergence is a measure of discrepancy between two probability distributions that belongs to the class of statistical distance measures, distinguished by its foundation in information theory and its quantification of the relative entropy or inefficiency arising from assuming one distribution over another.

491

Publications

40.8K

Citations

830

Authors

499

Institutions

Publications per year

2017–2026

124

Variants

Information-theoretic Divergence

Parents

Publications

491

Divergence measures based on the Shannon entropy

Jinfeng Lin · IEEE Transactions on Information Theory · 1991 · 5K citations

, , +13

$I$-Divergence Geometry of Probability Distributions and Minimization Problems

Imre Csiszár · The Annals of Probability · 1975 · 1.6K citations · Full text

, , +14

Rényi Divergence and Kullback-Leibler Divergence

Tim van Erven, Peter Harremoës · IEEE Transactions on Information Theory · 2014 · 1.5K citations · Full text

, , +12

An information-theoretic perspective of tf–idf measures

Akiko Aizawa · Information Processing & Management · 2002 · 1.4K citations

, , +13

Information-type measures of difference of probability distributions and indirect observation

Imre Csiszár · Studia Scientiarum Mathematicarum Hungarica · 1967 · 1.4K citations

, , +10

1–5 of 491

Authors

830

  1. PH
    Peter Harremoës

    Centrum Wiskunde & Informatica, Netherlands

    11 publications · 2.2K citations

    11

    h-index

  2. IV
    Igor Vajda

    Czech Academy of Sciences, Institute of Information Theory and Automation, Czechia

    11 publications · 954 citations

    11

    h-index

  3. AO
    Alfred O. Hero

    University of Michigan, United States

    10 publications · 546 citations

    10

    h-index

  4. FN
    Frank Nielsen

    Sony Computer Science Laboratories, Japan

    10 publications · 464 citations

    10

    h-index

  5. IJ
    Inder J. Taneja

    Universidade Federal de Santa Catarina, Brazil

    7 publications · 162 citations

    7

    h-index

1–5 of 830

Institutions

499

  1. Czech Academy of Sciences, Institute of Information Theory and Automation

    Prague, Czechia

    11 h-index

    11 publications

    920 citations

  2. University of Michigan

    Ann Arbor, United States

    10 h-index

    14 publications

    568 citations

  3. Sony Computer Science Laboratories

    Tokyo, Japan

    9 h-index

    9 publications

    558 citations

  4. University of Maryland, Baltimore County

    Baltimore, United States

    8 h-index

    9 publications

    1.5K citations

  5. Princeton University

    Princeton, United States

    8 h-index

    10 publications

    777 citations

  6. Technion – Israel Institute of Technology

    Haifa, Israel

    8 h-index

    10 publications

    976 citations

  7. Centre National de la Recherche Scientifique

    Paris, France

    7 h-index

    11 publications

    259 citations

  8. Consejo Nacional de Investigaciones Científicas y Técnicas

    Buenos Aires, Argentina

    7 h-index

    18 publications

    986 citations

1–8 of 499

Venues

  1. IEEE Transactions on Information Theory

    53 publications · 13K citations

    33

    h-index

  2. Entropy

    27 publications · 1.5K citations

    20

    h-index

  3. arXiv (Cornell University)

    Repository · 15 publications · 417 citations

    12

    h-index

  4. Information Sciences

    10 publications · 364 citations

    10

    h-index

  5. 7

    h-index

1–5