ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

TLDR

Modeling sentence pairs is critical for NLP tasks such as answer selection, paraphrase identification, and textual entailment, yet prior work typically fine‑tunes task‑specific systems, represents sentences independently, or relies on manually engineered features. The authors propose a general Attention‑Based Convolutional Neural Network (ABCNN) to model sentence pairs across multiple NLP tasks. ABCNN incorporates three attention mechanisms that embed mutual influence between sentences into convolutional layers, enabling each sentence representation to consider its counterpart. The interdependent representations produced by ABCNN outperform isolated sentence representations and achieve state‑of‑the‑art results on answer selection, paraphrase identification, and textual entailment, with code available at the provided GitHub link.

Abstract

How to model a pair of sentences is a critical issue in many NLP tasks such as answer selection (AS), paraphrase identification (PI) and textual entailment (TE). Most prior work (i) deals with one individual task by fine-tuning a specific system; (ii) models each sentence’s representation separately, rarely considering the impact of the other sentence; or (iii) relies fully on manually designed, task-specific linguistic features. This work presents a general Attention Based Convolutional Neural Network (ABCNN) for modeling a pair of sentences. We make three contributions. (i) The ABCNN can be applied to a wide variety of tasks that require modeling of sentence pairs. (ii) We propose three attention schemes that integrate mutual influence between sentences into CNNs; thus, the representation of each sentence takes into consideration its counterpart. These interdependent sentence pair representations are more powerful than isolated sentence representations. (iii) ABCNNs achieve state-of-the-art performance on AS, PI and TE tasks. We release code at: https://github.com/yinwenpeng/Answer_Selection .