2018 · 279 citations · 21 references
HumanitiesComputer ScienceTranslation StudiesComputational LinguisticsNatural Language ProcessingTranslation TheoryMachine TranslationTranslation Error AnalysisTranslation Quality AssessmentDocument-level EvaluationEvaluation ProtocolsHuman Evaluation Of Machine TranslationHuman Translation ComparisonMachine Translation Error AnalysisMachine Translation Quality EvaluationNeural Machine TranslationTranslation Accuracy Evaluation
Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese-English news translation task. We empirically test this claim with alternative evaluation protocols, contrasting the evaluation of single sentences and entire documents. In a pairwise ranking experiment, human raters assessing adequacy and fluency show a stronger preference for human over machine translation when evaluating documents as compared to isolated sentences. Our findings emphasise the need to shift towards document-level evaluation as machine translation improves to the degree that errors which are hard or impossible to spot at the sentence-level become decisive in discriminating quality of different translation outputs.
21
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Sequence to Sequence Learning with Neural Networks
Ilya Sutskever, Oriol Vinyals, Quoc V. Le · arXiv (Cornell University) · 2014 · 13.4K citations · Full text
Computing, Computer Science, French +14
Crowdsourcing user studies with Mechanical Turk
Aniket Kittur, Ed H., Bongwon Suh · 2008 · 2K citations