Overview of QA4MRE Main Task at CLEF 2013

Abstract

Abstract. This paper describes the Question Answering for Machine Reading (QA4MRE) Main Task at the 2013 Cross Language Evaluation Forum. In the main task, systems answered multiple-choice questions on documents con-cerned with four different topics. There were also two pilot tasks, Machine Reading on Biomedical Texts about Alz-heimer's disease, and Japanese Entrance Exams. This paper describes the preparation of the data sets, the definition of the background collections, the metric used for the evaluation of the systems ’ submissions, and the results. We intro-duced two novelties this year: auxiliary questions to evaluate systems level of inference, and a portion of questions where none of the options were correct. Nineteen groups participated in the task submitting a total of 77 runs in five languages. 1