Concepedia

Inter-annotator Agreement

Inter-Annotator Agreement is a measurement metric used in image annotation that quantifies the degree of consensus among multiple human annotators in assigning labels to visual data, distinguishing itself from other evaluation metrics by focusing specifically on the reliability and consistency of annotations rather than model performance.

490

Publications

27.4K

Citations

1.8K

Authors

623

Institutions

Publications per year

2017–2026

195

Variants

Iaa

Parents

Publications

490

SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation

Felix Hill, Roi Reichart, Anna Korhonen · Computational Linguistics · 2015 · 1.2K citations · Full text

, , +15

Learning From Crowds

Vikas C. Raykar, Shipeng Yu, Linda Zhao et al. · Journal of Machine Learning Research · 2010 · 1K citations

, , +13

NCBI disease corpus: A resource for disease name recognition and concept normalization

Rezarta Islamaj, Robert Leaman, Zhiyong Lu · Journal of Biomedical Informatics · 2014 · 837 citations · Full text

, , +17

Automatic Evaluation of Topic Coherence

David Newman, Jey Han Lau, Karl Grieser et al. · 2010 · 730 citations

, , +15

The Kappa Statistic: A Second Look

Barbara Di Eugenio, Michael Glass · Computational Linguistics · 2004 · 498 citations · Full text

, , +12

1–5 of 490

Authors

1.8K

  1. IG
    Iryna Gurevych

    Technische Universität Darmstadt, Germany

    9 publications · 502 citations

    10

    h-index

  2. MP
    Martha Palmer

    University of Colorado Boulder, United States

    7 publications · 510 citations

    7

    h-index

  3. LA
    Lora Aroyo

    Vrije Universiteit Amsterdam, Netherlands

    7 publications · 178 citations

    7

    h-index

  4. CW
    Chris Welty

    IBM (United States), United States

    5 publications · 97 citations

    5

    h-index

  5. JW
    Janyce Wiebe

    University of Pittsburgh, United States

    5 publications · 255 citations

    5

    h-index

1–5 of 1.8K

Institutions

623

  1. University of Pennsylvania

    Philadelphia, United States

    12 h-index

    23 publications

    3.4K citations

  2. University of Cambridge

    Cambridge, United Kingdom

    11 h-index

    32 publications

    4.4K citations

  3. University of Southern California

    Los Angeles, United States

    10 h-index

    58 publications

    2.8K citations

  4. Columbia University

    New York, United States

    10 h-index

    20 publications

    793 citations

  5. Johns Hopkins University

    Baltimore, United States

    9 h-index

    15 publications

    1.2K citations

  6. Centre National de la Recherche Scientifique

    Paris, France

    9 h-index

    26 publications

    1.2K citations

  7. University of Pittsburgh

    Pittsburgh, United States

    9 h-index

    21 publications

    988 citations

  8. University of Edinburgh

    Edinburgh, United Kingdom

    9 h-index

    22 publications

    2.7K citations

1–8 of 623

Venues

  1. arXiv (Cornell University)

    Repository · 20 publications · 540 citations

    13

    h-index

  2. Computational Linguistics

    10 publications · 2.2K citations

    11

    h-index

  3. Language Resources and Evaluation

    9 publications · 556 citations

    9

    h-index

  4. PubMed

    Repository · 9 publications · 440 citations

    9

    h-index

  5. Journal of the American Medical Informatics Association

    JAMIA · 8 publications · 490 citations

    9

    h-index

1–5