Contextual-CNN: A Novel Architecture Capturing Unified Meaning for Sentence Classification

Abstract

In this paper, we focus on the architecture of the convolutional neural network (CNN) for sentence classification. For understanding natural language, context in the sentence is important information for grasping the word sense. However, traditional CNN's feed-forward architecture is insufficient to reflect this factor. To solve this limitation, we propose a contextual CNN (C-CNN) for better text understanding by adding recurrent connection to the convolutional layer. This architecture helps CCNN units to be modulated over time with their neighboring units, thus the model integrates word meanings with surrounding information within the same layer. We evaluate our model on sentence-level sentiment prediction tasks and question categorization task. The C-CNN achieves state-of-the-art performances on fine-grained sentiment prediction and question categorization.