1994 · 90 citations · 9 references
Language StudiesComputer ScienceLinguisticsComputational LinguisticsMorphologyNatural Language ProcessingLanguage ModelingMorphological AnalysisMorphological ParsingMorphological SegmentationParsingText SegmentationJapanese Word SegmentationMorphological Parsing ModelsOpen TextStatistical Morphological Parsing
Japanese text lacks delimiters, making segmentation challenging, so a delimiter‑free method is needed. The study introduces a novel method for segmenting sentences into words and assigning parts of speech. The method employs a statistical language model and an efficient two‑pass N‑best search algorithm that does not require delimiters between words. The analyzer achieved 95.1 % recall and 94.6 % precision on the ATR Corpus.
We present a novel method for segmenting the input sentence into words and assigning parts of speech to the words. It consists of a statistical language model and an efficient two-pass N-best search algorithm. The algorithm does not require delimiters between words. Thus it is suitable for written Japanese. The proposed Japanese morphological analyzer achieved 95.1% recall and 94.6% precision for open text when it was trained and tested on the ATR Corpus.
9
A practical part-of-speech tagger
Doug Cutting, Julian Kupiec, Jan Pedersen et al. · 1992 · 632 citations · Full text
Procedure for quantitatively comparing the syntactic coverage of English grammars
Judith L. Klavans, Mark Liberman, Mitchell P. Marcus et al. · 1991 · 436 citations
Equations for part-of-speech tagging
Eugene Charniak, Curtis Hendrickson, Neil G. Jacobson et al. · 1993 · 169 citations