Recurrent Neural Network for Text Classification with Multi-Task Learning

TLDR

Neural network based methods have achieved great progress on many NLP tasks, yet most prior work relies on single-task objectives that suffer from insufficient training data. This paper adopts a multi-task learning framework to jointly learn across multiple related tasks. Using recurrent neural networks, we propose three sharing mechanisms with task-specific and shared layers and train the entire network jointly on all tasks. Experiments on four benchmark text classification tasks demonstrate that the proposed models improve a task’s performance by leveraging related tasks.

Abstract

Neural network based methods have obtained great progress on a variety of natural language processing tasks. However, in most previous works, the models are learned based on single-task supervised objectives, which often suffer from insufficient training data. In this paper, we use the multi-task learning framework to jointly learn across multiple related tasks. Based on recurrent neural network, we propose three different mechanisms of sharing information to model text with task-specific and shared layers. The entire network is trained jointly on all these tasks. Experiments on four benchmark text classification tasks show that our proposed models can improve the performance of a task with the help of other related tasks.