Convolutional Neural Networks for Sentence Classification

TLDR

The study reports experiments with convolutional neural networks built on pre‑trained word vectors for sentence classification and proposes a simple modification to combine static and task‑specific embeddings. The authors train CNNs on pre‑trained word vectors, fine‑tune task‑specific vectors, and modify the architecture to allow both static and task‑specific embeddings to be used. A simple CNN with static vectors achieves excellent results on multiple benchmarks, fine‑tuning task‑specific vectors yields further gains, and the models improve the state of the art on four of seven tasks, including sentiment analysis and question classification.

Abstract

We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the architecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification.