Generating query substitutions

TLDR

Query expansion via pseudo‑relevance feedback is costly and can cause query drift, while boolean or TFIDF‑based relaxation reduces query specificity. The paper introduces query substitution, generating a new query to replace the user’s original search query. The method generates substitutions by applying typical user modifications to the original query, evaluates them with a defined scale, and selects candidates using a feature‑based model trained on human relevance judgments. The approach yields high‑quality substitutions, improving candidate quality and significantly increasing coverage and effectiveness in sponsored search.

Abstract

We introduce the notion of query substitution, that is, generating a new query to replace a user's original search query. Our technique uses modifications based on typical substitutions web searchers make to their queries. In this way the new query is strongly related to the original query, containing terms closely related to all of the original terms. This contrasts with query expansion through pseudo-relevance feedback, which is costly and can lead to query drift. This also contrasts with query relaxation through boolean or TFIDF retrieval, which reduces the specificity of the query. We define a scale for evaluating query substitution, and show that our method performs well at generating new queries related to the original queries. We build a model for selecting between candidates, by using a number of features relating the query-candidate pair, and by fitting the model to human judgments of relevance of query suggestions. This further improves the quality of the candidates generated. Experiments show that our techniques significantly increase coverage and effectiveness in the setting of sponsored search.