Concepedia

Document Deduplication

Document Deduplication is a data curation process that identifies and removes redundant or duplicate document instances within a dataset while preserving a single canonical representation, distinguishing it from other data preprocessing methods by its exclusive focus on eliminating verbatim or near-identical document-level repetitions.

7

Publications

243

Citations

37

Authors

17

Institutions

Publications per year

2017–2026

3

Parents

Publications

7

Reducing systematic review burden using Deduklick: a novel, automated, reliable, and explainable deduplication algorithm to foster medical research

Nikolay Borissov, Quentin Haas, Beatrice Minder et al. · Systematic Reviews · 2022 · 105 citations · Full text

, , +17

Rule-based deduplication of article records from bibliographic databases

Yu Jiang, Can Lin, Weiyi Meng et al. · Database · 2014 · 40 citations · Full text

, , +16

Unsupervised deduplication using cross-field dependencies

Rob Hall, Charles Sutton, Andrew McCallum · 2008 · 31 citations · Full text

, , +14

An unsupervised heuristic-based approach for bibliographic metadata deduplication

Eduardo Nunes Borges, Moisés G. de Carvalho, Renata Galante et al. · Information Processing & Management · 2011 · 25 citations

, , +14

CopyCat: Near-Duplicates Within and Between the ClueWeb and the Common Crawl

Maik Fröbe, Janek Bevendorff, Lukas Gienapp et al. · 2021 · 19 citations

, , +9

1–5 of 7

Authors

37

  1. HJ
    Heidrun Janka

    HES-SO Genève, Switzerland

    1 publications · 105 citations

    1

    h-index

  2. MP
    Martin Potthast

    Bauhaus-Universität Weimar, Germany

    1 publications · 19 citations

    1

    h-index

  3. LA
    Lucas Anastasiou

    The Open University, United Kingdom

    1 publications · 10 citations

    1

    h-index

  4. BM
    Beatrice Minder

    University of Bern, Switzerland

    1 publications · 105 citations

    1

    h-index

  5. DT
    Douglas Teodoro

    University of Geneva, Switzerland

    1 publications · 105 citations

    1

    h-index

1–5 of 37

Institutions

17

  1. The Open University

    Milton Keynes, United Kingdom

    1 h-index

    3 publications

    30 citations

  2. University of Massachusetts Amherst

    Amherst Center, United States

    1 h-index

    3 publications

    93 citations

  3. Risk Engineering (Bulgaria)

    Sofia, Bulgaria

    1 h-index

    1 publications

    105 citations

  4. University of Illinois Chicago

    Chicago, United States

    1 h-index

    6 publications

    240 citations

  5. Oregon Health & Science University

    Portland, United States

    1 h-index

    6 publications

    240 citations

  6. Universidade Federal de Minas Gerais

    Belo Horizonte, Brazil

    1 h-index

    3 publications

    75 citations

  7. University of Passau

    Passau, Germany

    1 h-index

    1 publications

    13 citations

  8. HES-SO Genève

    Geneva, Switzerland

    1 h-index

    1 publications

    105 citations

1–8 of 17

Venues

  1. Systematic Reviews

    1 publications · 105 citations

    1

    h-index

  2. Open Research Online (The Open University)

    Repository · 1 publications · 10 citations

    1

    h-index

  3. Database

    1 publications · 40 citations

    1

    h-index

  4. Information Processing & Management

    1 publications · 25 citations

    1

    h-index

  5. Frontiers in Drug Safety and Regulation

    1 publications · 14 citations

1–5