2004 · 270 citations · 19 references
Syntactic ParsingEngineeringMachine LearningData PreparationRewrite PatternsText MiningData GenerationNatural Language ProcessingSyntaxInformation RetrievalData ScienceData MiningComputational LinguisticsBleu MeasureLanguage EngineeringData IntegrationGrammarLanguage StudiesData ManagementStatisticsRewriting SystemMachine TranslationSequence ModellingStatistical Mt SystemNlp TaskKnowledge DiscoveryComputer ScienceData ManipulationRetrieval Augmented GenerationRegulated RewritingLinguistic PhrasesLinguisticsData Modeling
Clump‑based statistical MT systems lack mechanisms for generalizing reorderings and fail to respect linguistic phrase boundaries. The study proposes using automatically learned rewrite patterns to preprocess source sentences, aligning their word order with the target language. The hybrid system combines a statistical model with broad‑coverage rule‑based parsers used during training to learn rewrite patterns and at runtime to reorder source sentences. Experiments demonstrate a 10 % relative improvement in BLEU score.
Current clump-based statistical MT systems have two limitations with respect to word ordering: First, they lack a mechanism for expressing and using generalization that accounts for reorderings of linguistic phrases. Second, the ordering of target words in such systems does not respect linguistic phrase boundaries. To address these limitations, we propose to use automatically learned rewrite patterns to preprocess the source sentences so that they have a word order similar to that of the target language. Our system is a hybrid one. The basic model is statistical, but we use broad-coverage rule-based parsers in two ways - during training for learning rewrite patterns, and at runtime for reordering the source sentences. Our experiments show 10% relative improvement in Bleu measure.
19
Kishore Papineni, Salim Roukos, Todd J. Ward et al. · 2001 · 20.9K citations · Full text
Natural Language Processing, Computer-assisted Translation, Engineering +10
The mathematics of statistical machine translation: parameter estimation
Peter F. Brown, Vincent J. Della Pietra, Stephen A. Della Pietra et al. · 1993 · 4.1K citations