Cosmos QA: Machine Reading Comprehension with Contextual Commonsense\n Reasoning

Abstract

Understanding narratives requires reading between the lines, which in turn,\nrequires interpreting the likely causes and effects of events, even when they\nare not mentioned explicitly. In this paper, we introduce Cosmos QA, a\nlarge-scale dataset of 35,600 problems that require commonsense-based reading\ncomprehension, formulated as multiple-choice questions. In stark contrast to\nmost existing reading comprehension datasets where the questions focus on\nfactual and literal understanding of the context paragraph, our dataset focuses\non reading between the lines over a diverse collection of people's everyday\nnarratives, asking such questions as "what might be the possible reason of\n...?", or "what would have happened if ..." that require reasoning beyond the\nexact text spans in the context. To establish baseline performances on Cosmos\nQA, we experiment with several state-of-the-art neural architectures for\nreading comprehension, and also propose a new architecture that improves over\nthe competitive baselines. Experimental results demonstrate a significant gap\nbetween machine (68.4%) and human performance (94%), pointing to avenues for\nfuture research on commonsense machine comprehension. Dataset, code and\nleaderboard is publicly available at https://wilburone.github.io/cosmos.\n