2010 · 35 citations · 29 references
ComputingComputer ScienceInformation RetrievalNatural Language ProcessingText AnalyticsDocument RepresentationLanguage ModelingRetrieval ModelsBm25Document GenerationDocument RelevanceDocument Relevance ModelingRevision History AnalysisTerm FrequencyText Retrieval
The generative process underlies many information retrieval models, notably statistical language models. Yet these models only examine one (current) version of the document, effectively ignoring the actual document generation process. We posit that a considerable amount of information is encoded in the document authoring process, and this information is complementary to the word occurrence statistics upon which most modern retrieval models are based. We propose a new term weighting model, Revision History Analysis (RHA), which uses the revision history of a document (e.g., the edit history of a page in Wikipedia) to redefine term frequency - a key indicator of document topic/relevance for many retrieval models and text processing tasks. We then apply RHA to document ranking by extending two state-of-the-art text retrieval models, namely, BM25 and the generative statistical language model (LM). To the best of our knowledge, our paper is the first attempt to directly incorporate document authoring history into retrieval models. Empirical results show that RHA provides consistent improvements for state-of-the-art retrieval models, using standard retrieval tasks and benchmarks.
29
Stephen Robertson, Steve Walker, Susan Jones et al. · Text REtrieval Conference · 1994 · 2.2K citations
LDA-based document models for ad-hoc retrieval
Xing Wei, W. Bruce Croft · 2006 · 1.1K citations
Information diffusion through blogspace
Daniel Gruhl, R. Guha, David Liben‐Nowell et al. · 2004 · 1.1K citations