Journal of the American Society for Information Science · 1995 · 53 citations · 0 references
Language StudiesComputer ScienceLinguisticsComputational LinguisticsInformation RetrievalNatural Language ProcessingMorphological AnalysisMorphological ParsingMorphological SegmentationParsingSyntactic AnalysisText SegmentationAutomatic Text SegmentationConcatenationFull-text RetrievalMorpheme Boundary DetectionPhrase BoundariesSyntactic DisambiguationText RetrievalWord Segmentation
Chinese text segmentation is essential for retrieval but is challenging because Chinese lacks explicit word boundaries, a problem that inspired the authors to adapt English morpheme‑ and phrase‑analysis techniques. The study presents ACTS, an automatic Chinese text segmentation prototype for full‑text retrieval, based on a hypothesis that multi‑character words and phrases can be modeled by a grammar of morphological and syntactic concatenation. ACTS performs partial syntactic analysis by first segmenting text into one‑ and two‑character units using a dictionary, then disambiguating their syntactic categories based on context, and finally parsing them with a grammar that models the concatenation of morphological and syntactic categories to form compound and complex words for indexing. Experiments on 30 texts demonstrate that ACTS extracts most significant words and phrases with high accuracy. © 1995 John Wiley & Sons, Inc.
Text segmentation is a prerequisite for text retrieval systems. Chinese texts cannot be readily segmented into words because they do not contain word boundaries. ACTS is an automatic Chinese text segmentation proto-type for Chinese full text retrieval. It applies partial syntactic analysis—the analysis of morphemes, words, and phrases. The idea was originally largely inspired by experiments on English morpheme and phrase-analysis-based text retrieval, which are particularly germane to Chinese, because neither Chinese nor English texts have morpheme and phrase boundaries. ACTS is built on the hypothesis that Chinese words and phrases exceeding two characters can be characterized by a grammar that describes the concatenation behavior of the morphological and syntactic categories of their formatives. This is examined through three procedures: (1) Segmentation—texts are divided into one and two character segments by matching against a dictionary; (2) Category disambiguation—the syntactic categories of segments are determined according to context; (3) Parsing—the segments are analyzed based on the grammar, and subsequently combined into compound and complex words for indexing and retrieval. The experimental results, based on a small sample of 30 texts, show that most significant words and phrases in these texts can be extracted with a high degree of accuracy. © 1995 John Wiley & Sons, Inc.