Synthetic and Natural Noise Both Break Neural Machine Translation

Yonatan Belinkov, Yonatan Bisk
11/6/2017
cs.CLcs.LG

Official code: ybisk/charNMT-noise (29 stars, last commit 2018-04-27).

Abstract

Character-based neural machine translation (NMT) models alleviate out-of-vocabulary issues, learn morphology, and move us closer to completely end-to-end translation systems. Unfortunately, they are also very brittle and easily falter when presented with noisy data. In this paper, we confront NMT models with synthetic and natural sources of noise. We find that state-of-the-art models fail to translate even moderately noisy texts that humans have no trouble comprehending. We explore two approaches to increase model robustness: structure-invariant word representations and robust training on noisy texts. We find that a model based on a character convolutional neural network is able to simultaneously learn representations robust to multiple kinds of noise.

View on arXivView PDF

Code Implementations(1)

ybisk/charNMT-noiseOfficial100%

Scripts and noise data for Belinkov & Bisk 2018

298Lua, PythonFeb 2, 20188 years ago

Similar Papers

Did Translation Models Get More Robust Without Anyone Even Noticing?

90% match2024

Ask Language Model to Clean Your Noisy Translation Data

89% match2023

Is Robustness Transferable across Languages in Multilingual Neural Machine Translation?

88% match2023

INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation

87% match1 repo142023

Fully Character-Level Neural Machine Translation without Explicit Segmentation

86% match1 repo1432016

Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translation

86% match1 repo32023

Cite this paper

@article{belinkov2017synthetic,
  title  = {Synthetic and Natural Noise Both Break Neural Machine Translation},
  author = {Yonatan Belinkov and Yonatan Bisk},
  year   = {2017},
  eprint = {1711.02173},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1711.02173}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1711.02173.svg)](https://codeofpaper.com/paper/1711.02173)

Discussion