Overview of QA4MRE at CLEF 2011: Question Answering for Machine Reading Evaluation

Abstract

Abstract. This paper describes the Question Answering for Machine Reading (QA4MRE) task at the 2012 Cross Language Evaluation Forum. In the main task, systems answered multiple-choice questions on documents concerned with four different topics. There were also two pilot tasks, Processing Modality and Negation for Machine Reading, and Machine Reading on Biomedical Texts about Alzheimer's disease. This paper describes the preparation of the data sets, the definition of the background collections, the metric used for the evaluation of the systems ’ submissions, and the results. Eleven groups participated in the task submitting a total of 43 runs in seven languages. 1.