Vector-space ranking with effective early termination

TLDR

Research has focused on improving information retrieval effectiveness through techniques such as relevance feedback, thesaural expansion, and pivoting, but these enhancements increase query evaluation costs. The paper investigates how to improve the cost‑effectiveness of searching. The authors propose an inverted file structure with quantized weights that, when combined with early termination heuristics, yields superior retrieval effectiveness over conventional structures. The new structure achieves comparable effectiveness at lower computational cost, offering a better cost/performance trade‑off than prior inverted file organizations.

Abstract

Considerable research effort has been invested in improving the effectiveness of information retrieval systems. Techniques such as relevance feedback, thesaural expansion, and pivoting all provide better quality responses to queries when tested in standard evaluation frameworks. But such enhancements can add to the cost of evaluating queries. In this paper we consider the pragmatic issue of how to improve the cost-effectiveness of searching. We describe a new inverted file structure using quantized weights that provides superior retrieval effectiveness compared to conventional inverted file structures when early termination heuristics are employed. That is, we are able to reach similar effectiveness levels with less computational cost, and so provide a better cost/performance compromise than previous inverted file organisations.