Fine-Grained Human Evaluation of Neural Versus Phrase-Based Machine Translation-Reference-Cited by-同舟云学术

Fine-Grained Human Evaluation of Neural Versus Phrase-Based Machine Translation

Published:2017-06-01 Issue:1 Volume:108 Page:121-132
ISSN:1804-0462
Container-title:The Prague Bulletin of Mathematical Linguistics
language:
Short-container-title:

Author:

Klubička Filip¹,Toral Antonio²,Sánchez-Cartagena Víctor M.³

Affiliation:

1. University of Zagreb

2. University of Groningen

3. Prompsit Language Engineering

Abstract

AbstractWe compare three approaches to statistical machine translation (pure phrase-based, factored phrase-based and neural) by performing a fine-grained manual evaluation via error annotation of the systems’ outputs. The error types in our annotation are compliant with the multidimensional quality metrics (MQM), and the annotation is performed by two annotators. Inter-annotator agreement is high for such a task, and results show that the best performing system (neural) reduces the errors produced by the worst system (phrase-based) by 54%.

Publisher

Walter de Gruyter GmbH

Subject

General Engineering

Link

https://content.sciendo.com/downloadpdf/journals/pralin/108/1/article-p121.xml

Reference22 articles.

1. Dealing with Data Sparseness in SMT with Factured Models and Morphological Expansion : a Case Study on of Modern;Sánchez;Journal Computing,2016

2. Dealing with Data Sparseness in SMT with Factured Models and Morphological Expansion : a Case Study on of Modern;Sánchez;Journal Computing,2016

3. Alexandra Neural machine translation of rare words with subword units arXiv preprint arXiv;Sennrich,1508

4. Josh evaluation of machine translation InProceedings of the Second Workshop on Statistical Machine Translation pages Association for;Callison;Meta Computational Linguistics,2007

5. Neural versus Phrase - Based Machine Translation Quality : a Case Study InProceedings of the Conference on Empirical Methods in Natural pages Association for;Bentivogli;Language Processing Computational Linguistics,2016

Cited by 54 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. An Analysis of Error Types in Chinese to English Translation by Google Neural Machine Translation;Proceedings of the 2023 International Joint Conference on Robotics and Artificial Intelligence;2023-07-07

2. Research on Machine Translation (MT) System Based on Deep Reinforcement Learning;2022 3rd International Conference on Computer Science and Management Technology (ICCSMT);2022-11

3. Application Design of Linguistics in Computer Technology under Artificial Intelligence Background;Mobile Information Systems;2022-09-28

4. La traduction automatique des textes faciles à lire et à comprendre (FALC) : une étude comparative;Meta;2022-09-07

5. Extralinguistic Consultation in English–Chinese Translation: A Study Drawing on Eye-Tracking and Screen-Recording Data;Frontiers in Psychology;2022-06-20