2012 · 25 citations · 14 references
ComputingComputer ScienceComputational LinguisticsInformation RetrievalNatural Language ProcessingLanguage AdaptationLanguage ModelingMachine TranslationText NormalizationText PreprocessingAcquisition TechniquesDomain AdaptationNormalizationStatistical Machine TranslationStatistical Text Normalization
This paper reports a set of domain adaptation techniques for improving Statistical Machine Translation (SMT) for usergenerated web forum content. We investigate both normalization and supplementary training data acquisition techniques, all guided by the aim of reducing the number of Out-Of-Vocabulary (OOV) items in the target language with respect to the training data. We classify OOVs into a set of types, and address each through dedicated normalization and/or supplementary training material selection-based approaches. We investigate the effect of these methods both in an additive as well as a contrastive scenario. Our findings show that (i) normalization and supplementary training material techniques can be complementary, (ii) for general forum data, fully automatic supplementary training data acquisition can perform as well or sometimes better than semi-automatic normalization (although tackling different types of OOVs) and (iii) for very noisy data, normalization really pays off.
14
Kishore Papineni, Salim Roukos, Todd J. Ward et al. · 2001 · 21.5K citations · Full text
Philipp Koehn, Richard Zens, Chris Dyer et al. · 2007 · 4.9K citations · Full text
A Systematic Comparison of Various Statistical Alignment Models
Franz Josef Och, Hermann Ney · Computational Linguistics · 2003 · 3.9K citations · Full text