ACM Transactions on Information Systems · 2013 · 23 citations · 27 references
ComputingComputer ScienceComputational LinguisticsInformation RetrievalNatural Language ProcessingCorpus AnalysisRetrieval ModelsMismatchRetrievalRetrieval SystemsStemmingThematic CoherenceTrec Evaluation
Stemming is a widely used technique in information retrieval systems to address the vocabulary mismatch problem arising out of morphological phenomena. The major shortcoming of the commonly used stemmers is that they accept the morphological variants of the query words without considering their thematic coherence with the given query, which leads to poor performance. Moreover, for many queries, such approaches also produce retrieval performance that is poorer than no stemming, thereby degrading the robustness. The main goal of this article is to present corpus-based fully automatic stemming algorithms which address these issues. A set of experiments on six TREC collections and three other non-English collections containing news and web documents shows that the proposed query-based stemming algorithms consistently and significantly outperform four state of the art strong stemmers of completely varying principles. Our experiments also confirm that the robustness of the proposed query-based stemming algorithms are remarkably better than the existing strong baselines.
27
An algorithm for suffix stripping
Martin Porter · Program electronic library and information systems · 1980 · 8.1K citations
Word association norms, mutual information, and lexicography
Kenneth Church, Patrick Hanks · 1990 · 3.7K citations
Development of a Stemming Algorithm
Julie B. Lovins · Mech. Transl. Comput. Linguistics · 1968 · 1.1K citations