2005 · 173 citations · 17 references
Language StudiesComputer ScienceEnglishComputational LinguisticsMorphologyNatural Language ProcessingLanguage ModelingLanguage ModelsMorphological AnalysisMorphological SegmentationLexicon-based Morphological SegmentationMaximum A Posteriori EstimationMorphological LearningNatural Language InferenceUnsupervised Morphological SegmentationUnsupervised Segmentation
The study proposes an unsupervised algorithm to induce a simple morphological lexicon for natural languages. A probabilistic maximum‑a‑posteriori model builds hierarchical representations of morphs, stores meaning and form parameters that influence morph roles, and is applied to unsupervised morpheme segmentation of Finnish and English. The method achieves strong performance on Finnish and comparable results on English.
This work presents an algorithm for the unsupervised learning, or induction, of a simple morphology of a natural language. A probabilistic maximum a posteriori model is utilized, which builds hierarchical representations for a set of morphs, which are morpheme-like units discovered from unannotated text corpora. The induced morph lexicon stores parameters related to both the “meaning” and “form” of the morphs it contains. These parameters affect the role of the morphs in words. The model is implemented in a task of unsupervised morpheme segmentation of Finnish and English words. Very good results are obtained for Finnish and almost as good results are obtained in the English task.
17
Unsupervised discovery of morphemes
Mathias Creutz, Krista Lagus · 2002 · 363 citations · Full text
A stochastic finite-state word-segmentation algorithm for Chinese
Richard Sproat, Chilin Shih, William A. Gale et al. · 1994 · 290 citations · Full text