ACM Transactions on Information Systems · 1998 · 303 citations · 17 references
ComputingComputer ScienceSpanishComputational LinguisticsInformation RetrievalNatural Language ProcessingCorpus AnalysisCorpus RetrievalEnd-user AcceptanceStemmingVariant Forms
Stemming reduces variant word forms to common roots in IR, but existing methods ignore corpus‑specific language use, potentially causing retrieval failures. The study aims to develop a stemmer that leverages corpus‑based word variant cooccurrence statistics. The proposed approach modifies or creates a stemmer by applying cooccurrence statistics derived from the target corpus. Experiments on English newspaper, legal, and Spanish texts show the technique is viable and outperforms conventional morphology‑only stemmers.
Stemming is used in many information retrieval (IR) systems to reduce variant word forms to common roots. It is one of the simplest applications of natural-language processing to IR and is one of the most effective in terms of user acceptance and consistency, though small retrieval improvements. Current stemming techniques do not, however, reflect the language use in specific corpora, and this can lead to occasional serious retrieval failures. We propose a technique for using corpus-based word variant cooccurrence statistics to modify or create a stemmer. The experimental results generated using English newspaper and legal text and Spanish text demonstrate the viability of this technique and its advantages relative to conventional approaches that only employ morphological rules.
17
An algorithm for suffix stripping
Martin Porter · Program electronic library and information systems · 1980 · 8.1K citations
Word association norms, mutual information, and lexicography
Kenneth Church, Patrick Hanks · 1990 · 3.7K citations
Word association norms, mutual information, and lexicography
Kenneth Church, Patrick Hanks · 1989 · 1.1K citations · Full text
Query expansion using lexical-semantic relations
1994 · 873 citations
Abraham Bookstein · Information Processing & Management · 1990 · 709 citations