1997 · 393 citations · 8 references
ComputingComputer ScienceEnglishComputational LinguisticsCorpus LinguisticsNatural Language ProcessingCorpus-based Language LearningInformation ExtractionLanguage ModelingLanguage ModelsPart-of-speech TaggingSyntactic AnalysisDomain-specific ModelingMaximum Entropy EstimationMaximum Entropy ModelingMaximum Entropy TaggingPart-of-speech FeaturesSentence Boundary AnnotationSentence Boundary DetectionText Boundary Detection
The authors present a trainable model for detecting sentence boundaries in raw text. The model learns to classify punctuation marks as valid or invalid sentence boundaries from annotated corpora, requiring no hand‑crafted rules or domain‑specific resources, and can be trained on any English genre or other Roman‑alphabet language. The system achieves performance comparable to or better than similar systems, highlighting its ease of retraining for new domains.
We present a trainable model for identifying sentence boundaries in raw text. Given a corpus annotated with sentence boundaries, our model learns to classify each occurrence of., ?, and ! as either a valid or invalid sentence boundary. The training procedure requires no hand-crafted rules, lexica, part-of-speech tags, or domain-specific information. The model can therefore be trained easily on any genre of English, and should be trainable on any other Romanalphabet language. Performance is comparable to or better than the performance of similar systems, but we emphasize the simplicity of retraining for new domains.
8
A Maximum Entropy Model for Part-Of-Speech Tagging
Adwait Ratnaparkhi · 1996 · 1.3K citations
A practical part-of-speech tagger
Doug Cutting, Julian Kupiec, Jan Pedersen et al. · 1992 · 632 citations · Full text
A new statistical parser based on bigram lexical dependencies
Michael Collins · 1996 · 622 citations · Full text
Adaptive multilingual sentence boundary disambiguation
David D. Palmer, Marti A. Hearst · 1997 · 152 citations