Understanding Back-Translation at Scale

Sergey Edunov, Myle Ott, Michael Auli, David Grangier
8/28/2018
cs.CL

Official code: facebookresearch/fairseq (32,213 stars, MIT, PyTorch, last commit 2025-09-30).

Abstract

An effective method to improve neural machine translation with monolingual data is to augment the parallel training corpus with back-translations of target language sentences. This work broadens the understanding of back-translation and investigates a number of methods to generate synthetic source sentences. We find that in all but resource poor settings back-translations obtained via sampling or noised beam outputs are most effective. Our analysis shows that sampling or noisy synthetic data gives a much stronger training signal than data generated by beam or greedy search. We also compare how synthetic data compares to genuine bitext and study various domain effects. Finally, we scale to hundreds of millions of monolingual sentences and achieve a new state of the art of 35 BLEU on the WMT'14 English-German test set.

View on arXivView PDF

Code Implementations(1)

facebookresearch/fairseqOfficial100%
PyTorchMIT

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

32,2136,667C++, LuaAug 29, 20171 years agoMIT
artificial-intelligencepythonpytorch

Similar Papers

Non-Fluent Synthetic Target-Language Data Improve Neural Machine Translation

89% match1 repo12024

Phrase-Based & Neural Unsupervised Machine Translation

89% match1 repo1.5k2018

Unsupervised Neural Machine Translation

87% match2 repos2.3k2017

Boosting Unsupervised Machine Translation with Pseudo-Parallel Data

86% match2023

Data Augmentation With Back translation for Low Resource languages: A case of English and Luganda

86% match2025

Back-Translation Still Matters: Leveraging Monolingual Data for Low-Resource Machine Translation with LLMs

85% match2026

Cite this paper

@article{edunov2018understanding,
  title  = {Understanding Back-Translation at Scale},
  author = {Sergey Edunov and Myle Ott and Michael Auli and David Grangier},
  year   = {2018},
  eprint = {1808.09381},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1808.09381}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1808.09381.svg)](https://codeofpaper.com/paper/1808.09381)

Discussion