Achieving Human Parity on Automatic Chinese to English News Translation

Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark +19 more
3/15/2018
cs.CL

Official code: cfedermann/Appraise (77 stars, BSD-3-Clause, last commit 2021-05-07).

Abstract

Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate across language barriers. The question naturally arises whether such systems can approach or achieve parity with human translations. In this paper, we first address the problem of how to define and accurately measure human parity in translation. We then describe Microsoft's machine translation system and measure the quality of its translations on the widely used WMT 2017 news translation task from Chinese to English. We find that our latest neural machine translation system has reached a new state-of-the-art, and that the translation quality is at human parity when compared to professional human translations. We also find that it significantly exceeds the quality of crowd-sourced non-professional translations.

View on arXivView PDF

Code Implementations(1)

cfedermann/AppraiseOfficial100%
BSD-3-Clause

Appraise evaluation system for manual evaluation of machine translation output

7737CSS, HTMLOct 23, 20115 years agoBSD-3-Clause

Similar Papers

Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progress

86% match1 repo32025

Benchmarking GPT-4 against Human Translators: A Comprehensive Evaluation Across Languages, Domains, and Expertise Levels

86% match1 repo12024

Convergences and Divergences between Automatic Assessment and Human Evaluation: Insights from Comparing ChatGPT-Generated Translation and Neural Machine Translation

85% match2024

EvolveMT: an Ensemble MT Engine Improving Itself with Usage Only

85% match2023

Synslator: An Interactive Machine Translation Tool with Online Learning

85% match2023

No Language Left Behind: Scaling Human-Centered Machine Translation

84% match2 repos32.2k2022

Cite this paper

@article{hassan2018achieving,
  title  = {Achieving Human Parity on Automatic Chinese to English News Translation},
  author = {Hany Hassan and Anthony Aue and Chang Chen and Vishal Chowdhary and Jonathan Clark and Christian Federmann and Xuedong Huang and Marcin Junczys-Dowmunt and William Lewis and Mu Li and Shujie Liu and Tie-Yan Liu and Renqian Luo and Arul Menezes and Tao Qin and Frank Seide and Xu Tan and Fei Tian and Lijun Wu and Shuangzhi Wu and Yingce Xia and Dongdong Zhang and Zhirui Zhang and Ming Zhou},
  year   = {2018},
  eprint = {1803.05567},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1803.05567}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1803.05567.svg)](https://codeofpaper.com/paper/1803.05567)

Discussion