A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval

TLDR

The paper proposes a latent semantic model with convolutional‑pooling over word sequences to learn low‑dimensional semantic vectors for queries and documents. The model captures contextual word‑n‑gram features within a temporal window, aggregates salient n‑grams into sentence‑level vectors, applies a non‑linear transformation to extract high‑level semantics, and is trained on click‑through data for web document ranking. Results show the model captures salient semantic information and significantly outperforms previous state‑of‑the‑art semantic models on a large‑scale web document ranking task.

Abstract

In this paper, we propose a new latent semantic model that incorporates a convolutional-pooling structure over word sequences to learn low-dimensional, semantic vector representations for search queries and Web documents. In order to capture the rich contextual structures in a query or a document, we start with each word within a temporal context window in a word sequence to directly capture contextual features at the word n-gram level. Next, the salient word n-gram features in the word sequence are discovered by the model and are then aggregated to form a sentence-level feature vector. Finally, a non-linear transformation is applied to extract high-level semantic information to generate a continuous vector representation for the full text string. The proposed convolutional latent semantic model (CLSM) is trained on clickthrough data and is evaluated on a Web document ranking task using a large-scale, real-world data set. Results show that the proposed model effectively captures salient semantic information in queries and documents for the task while significantly outperforming previous state-of-the-art semantic models.