A Convolutional Neural Network for Modelling Sentences

TLDR

The ability to accurately represent sentences is central to language understanding. The authors propose the Dynamic Convolutional Neural Network (DCNN) for semantic modelling of sentences. The DCNN employs dynamic k‑max pooling to produce a global feature graph that captures short‑ and long‑range relations without relying on parse trees, enabling variable‑length input and multilingual applicability, and is evaluated on sentiment prediction, question classification, and Twitter sentiment tasks. The network achieves excellent performance on the first three tasks and reduces error by over 25 % on the Twitter sentiment task compared to the strongest baseline.

Abstract

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.