Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding

Guangyu Yang, Jinghong Chen, Weizhe Lin, Bill Byrne
11/14/2023
cs.CL

No official code found. Best implementation: BruceYg/DPO-MBR (10 stars, MIT, last commit 2025-08-05).

Abstract

Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Reinforcement Learning technique, Direct Preference Optimization (DPO), can fine-tune MLLMs to get the gains of MBR without any additional computation in inference. Our method uses only a small monolingual fine-tuning set and yields significantly improved performance on multiple NMT test sets compared to MLLMs without DPO.

View on arXivView PDF

Code Implementations(1)

BruceYg/DPO-MBRCommunity100%
MIT
100PythonApr 1, 20241 years agoMIT

Similar Papers

Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study

88% match2026

Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimization

86% match2024

Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation

86% match2026

PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matching

86% match2023

Simultaneous Machine Translation with Tailored Reference

85% match2023

Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation

85% match1 repo2026

Cite this paper

@article{yang2023direct,
  title  = {Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding},
  author = {Guangyu Yang and Jinghong Chen and Weizhe Lin and Bill Byrne},
  year   = {2023},
  eprint = {2311.08380},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.08380v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.08380.svg)](https://codeofpaper.com/paper/2311.08380)

Discussion