Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM

Rachel Bawden, François Yvon
3/3/2023
cs.CL

No official code found. Best implementation: rbawden/mt-bigscience (11 stars, last commit 2023-05-15).

Abstract

The NLP community recently saw the release of a new large open-access multilingual language model, BLOOM (BigScience et al., 2022) covering 46 languages. We focus on BLOOM's multilingual ability by evaluating its machine translation performance across several datasets (WMT, Flores-101 and DiaBLa) and language pairs (high- and low-resourced). Our results show that 0-shot performance suffers from overgeneration and generating in the wrong language, but this is greatly improved in the few-shot setting, with very good results for a number of language pairs. We study several aspects including prompt design, model sizes, cross-lingual transfer and the use of discursive context.

View on arXivView PDF

Code Implementations(1)

rbawden/mt-bigscienceCommunity100%

Evaluation results for Machine Translation within the BigScience project

112Shell, PythonDec 12, 20223 years ago

Similar Papers

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

91% match1 repo362023

ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning

90% match2023

BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages

90% match1 repo2272023

Omnilingual MT: Machine Translation for 1,600 Languages

90% match2026

When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages

90% match1 repo102023

Multilingual Text Representation

89% match2023

Cite this paper

@article{bawden2023investigating,
  title  = {Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM},
  author = {Rachel Bawden and François Yvon},
  year   = {2023},
  eprint = {2303.01911},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.01911v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.01911.svg)](https://codeofpaper.com/paper/2303.01911)

Discussion