Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?

Àlex R. Atrio, Alexis Allemann, Ljiljana Dolamic, Andrei Popescu-Belis
10/5/2024
cs.CL

No confident code implementation found for this paper.

Abstract

Many-to-one neural machine translation systems improve over one-to-one systems when training data is scarce. In this paper, we design and test a novel algorithm for selecting the language of minibatches when training such systems. The algorithm changes the language of the minibatch when the weights of the model do not evolve significantly, as measured by the smoothed KL divergence between all layers of the Transformer network. This algorithm outperforms the use of alternating monolingual batches, but not the use of shuffled batches, in terms of translation quality (measured with BLEU and COMET) and convergence speed.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Beyond Vanilla Fine-Tuning: Leveraging Multistage, Multilingual, and Domain-Specific Methods for Low-Resource Machine Translation

86% match2025

Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation

85% match2 repos1522020

Exploiting Domain-Specific Parallel Data on Multilingual Language Models for Low-resource Language Translation

84% match2024

Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learning

84% match2024

Segment-Based Interactive Machine Translation for Pre-trained Models

84% match2024

Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation

83% match2024

Cite this paper

@article{atrio2024variation,
  title  = {Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?},
  author = {Àlex R. Atrio and Alexis Allemann and Ljiljana Dolamic and Andrei Popescu-Belis},
  year   = {2024},
  eprint = {2410.04147},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.04147v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.04147.svg)](https://codeofpaper.com/paper/2410.04147)

Discussion