Stemming of French words based on grammatical categories

Jacques Savoy

Journal of the American Society for Information Science · 1993 · 82 citations · 9 references

Concepts

TL;DR

Suffix stripping algorithms cluster words sharing a root, but current methods operate either context‑free or context‑sensitive, relying only on the remaining stem. The authors propose a suffixing algorithm that incorporates grammatical categories to improve stemming. For French, the algorithm performs morphological analysis to remove inflectional suffixes, then applies a dictionary‑based suffix stripping guided by grammatical categories to eliminate derivational suffixes. The method yields linguistically correct lemmas with a mean error rate of 16 %, supports foreign languages, yet cannot achieve substantially better results. © 1993 John Wiley & Sons, Inc.

Abstract

Automatic indexing systems use suffix stripping algorithms to cluster various words derived from a common root under the same stem. Currently, removing affixes to either a context-free or context-sensitive operation, where the context refers to the remaining stem. In this article, we propose a suffixing algorithm which uses grammatical categories to enhance the stemming process. This approach supports the use of foreign languages. In our case, the language is French, and a morphological analysis is required for removing inflectional suffixes or morphosyntactic variants of a lemma. After this analysis, we implement a suffix stripping algorithm which uses a dictionary and the grammatical categories to remove derivational suffixes. Our approach always returns a linguistically correct lemma, but not necessarily the "right" one. Based on our tests, this solution is an attractive one, with a mean error rate of 16%. We finish by explaining why we cannot expect significantly better results with this approach. © 1993 John Wiley & Sons, Inc.

References

9