Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation

Shravan Nayak, Surangika Ranathunga, Sarubi Thillainathan, Rikki Hung, Anthony Rinaldi +4 more
6/2/2023
cs.CL

No confident code implementation found for this paper.

Abstract

NMT systems trained on Pre-trained Multilingual Sequence-Sequence (PMSS) models flounder when sufficient amounts of parallel data is not available for fine-tuning. This specifically holds for languages missing/under-represented in these models. The problem gets aggravated when the data comes from different domains. In this paper, we show that intermediate-task fine-tuning (ITFT) of PMSS models is extremely beneficial for domain-specific NMT, especially when target domain data is limited/unavailable and the considered languages are missing or under-represented in the PMSS model. We quantify the domain-specific results variations using a domain-divergence test, and show that ITFT can mitigate the impact of domain divergence to some extent.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Divergence-Based Domain Transferability for Zero-Shot Classification

86% match2023

Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast

85% match2023

Replay to Remember: Continual Layer-Specific Fine-tuning for German Speech Recognition

85% match2023

Where to Adapt Matters: Layer-Selective Fine-Tuning for Capability Retention

84% match2026

Fully Fine-tuned CLIP Models are Efficient Few-Shot Learners

84% match2024

Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Prompt

84% match1 repo52023

Cite this paper

@article{nayak2023leveraging,
  title  = {Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation},
  author = {Shravan Nayak and Surangika Ranathunga and Sarubi Thillainathan and Rikki Hung and Anthony Rinaldi and Yining Wang and Jonah Mackey and Andrew Ho and En-Shiun Annie Lee},
  year   = {2023},
  eprint = {2306.01382},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.01382v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.01382.svg)](https://codeofpaper.com/paper/2306.01382)

Discussion