2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text

TLDR

The 2010 i2b2/VA Workshop on NLP for Clinical Records introduced three tasks—concept extraction, assertion classification, and relation classification—to evaluate automated processing of medical reports. An annotated reference standard corpus was released, and 22 concept extraction, 21 assertion classification, and 16 relation classification systems were built and evaluated against it. Results showed that combining machine learning with rule‑based components, ensembles, unlabeled data, and external knowledge improves concept, assertion, and relation extraction, especially when training data are limited.

Abstract

Abstract The 2010 i2b2/VA Workshop on Natural Language Processing Challenges for Clinical Records presented three tasks: a concept extraction task focused on the extraction of medical concepts from patient reports; an assertion classification task focused on assigning assertion types for medical problem concepts; and a relation classification task focused on assigning relation types that hold between medical problems, tests, and treatments. i2b2 and the VA provided an annotated reference standard corpus for the three tasks. Using this reference standard, 22 systems were developed for concept extraction, 21 for assertion classification, and 16 for relation classification. These systems showed that machine learning approaches could be augmented with rule-based systems to determine concepts, assertions, and relations. Depending on the task, the rule-based systems can either provide input for machine learning or post-process the output of machine learning. Ensembles of classifiers, information from unlabeled data, and external knowledge sources can help when the training data are inadequate.