Sequence Level Training with Recurrent Neural Networks

TLDR

Many natural language processing applications use language models to generate text, typically trained to predict the next word given previous words and context such as an image, but at test time the model must generate the entire sequence from scratch, creating a discrepancy that makes generation brittle as errors accumulate. The study proposes a sequence‑level training algorithm that directly optimizes test‑time metrics such as BLEU or ROUGE. The algorithm trains recurrent neural networks by optimizing these metrics directly, aligning the training objective with generation evaluation. On three tasks, the approach outperforms strong baselines for greedy generation and remains competitive with beam search while being several times faster.

Abstract

Abstract: Many natural language processing applications use language models to generate text. These models are typically trained to predict the next word in a sequence, given the previous words and some context such as an image. However, at test time the model is expected to generate the entire sequence from scratch. This discrepancy makes generation brittle, as errors may accumulate along the way. We address this issue by proposing a novel sequence level training algorithm that directly optimizes the metric used at test time, such as BLEU or ROUGE. On three different tasks, our approach outperforms several strong baselines for greedy generation. The method is also competitive when these baselines employ beam search, while being several times faster.