A Study of Stemming Effects on Information Retrieval in Bahasa Indonesia

Fadillah Tala

2003 · 278 citations · 19 references

Concepts

TL;DR

Stemming maps morphological variants to a common stem, a process also called conflation, and is widely used in information retrieval to improve performance by assuming terms sharing a stem have similar meaning. This thesis investigates the effect of Bahasa Indonesia stemming algorithms on information retrieval, evaluating existing stemmers against a newly created rule‑based stemmer. The study compares existing stemmers with a rule‑based stemmer derived from morphological analysis, applying stemming at indexing time to reduce index size and enhance retrieval. Stemming improves retrieval performance and reduces the index file size.

Abstract

Stemming is a process which provides a mapping of different morphological variants of words into their base/common word (stem). This process is also known as conflation. Based on the assumption that terms which have a common stem will usually have similar meaning, the stemming process is widely used in Information Retrieval as a way to improve retrieval performance. In addition to its ability to improve the retrieval performance, the stemming process, which is done at indexing time, will also reduce the size of the index file. This thesis is about a study of stemming algorithms in Bahasa Indonesia, especially their effect on the information retrieval. We try to evaluate the existing stemmer for Bahasa Indonesia and compare it with a purely rule-based stemmer, which we created for this purpose. This rule-based stemmer is developed based on a study of morphological structure of Bahasa Indonesia words.

References

19