Word RNN as a Baseline for Sentence Completion

Abstract

Since sentence completion task requires diverse abilities including linguistic proficiency, common knowledge, and logical reasoning, the task has been used to measure the reading comprehension level of not only humans but also machines. This work presents a word Recurrent Neural Network (RNN), a popular neural language model, as a competitive baseline for sentence completion by improving the performance with an appropriate network structure and hyper-parameters. We also propose a bidirectional version of word RNN, which has been shown to give further improvement. In addition, we formulate and compare blank loss and sentence loss criteria for selecting the best option to complete the sentence with a trained word RNN. Despite its simple architecture, the accuracy results of our word RNNs exceed the previous best results on Microsoft Research Sentence Completion Challenge and the Scholastic Aptitude Test (SAT) sentence comnletion Questions.