Improving a statistical MT system with automatically learned rewrite patterns

Fei Xia, Michael C. McCord

2004 · 270 citations · 19 references

DOIFull text

Open access

Concepts

TL;DR

Clump‑based statistical MT systems lack mechanisms for generalizing reorderings and fail to respect linguistic phrase boundaries. The study proposes using automatically learned rewrite patterns to preprocess source sentences, aligning their word order with the target language. The hybrid system combines a statistical model with broad‑coverage rule‑based parsers used during training to learn rewrite patterns and at runtime to reorder source sentences. Experiments demonstrate a 10 % relative improvement in BLEU score.

Abstract

Current clump-based statistical MT systems have two limitations with respect to word ordering: First, they lack a mechanism for expressing and using generalization that accounts for reorderings of linguistic phrases. Second, the ordering of target words in such systems does not respect linguistic phrase boundaries. To address these limitations, we propose to use automatically learned rewrite patterns to preprocess the source sentences so that they have a word order similar to that of the target language. Our system is a hybrid one. The basic model is statistical, but we use broad-coverage rule-based parsers in two ways - during training for learning rewrite patterns, and at runtime for reordering the source sentences. Our experiments show 10% relative improvement in Bleu measure.

References

19