MIRO: MultI-Reward cOnditioned pretraining improves T2I quality and efficiency

Nicolas Dufour, Lucas Degeorge, Arijit Ghosh, Vicky Kalogeiton, David Picard
10/29/2025
cs.CVcs.LG

No official code found. Best implementation: nicolas-dufour/miro (40 stars, last commit 2025-10-31).

Abstract

Current text-to-image generative models are trained on large uncurated datasets to enable diverse generation capabilities. However, this does not align well with user preferences. Recently, reward models have been specifically designed to perform post-hoc selection of generated images and align them to a reward, typically user preference. This discarding of informative data together with the optimizing for a single reward tend to harm diversity, semantic fidelity and efficiency. Instead of this post-processing, we propose to condition the model on multiple reward models during training to let the model learn user preferences directly. We show that this not only dramatically improves the visual quality of the generated images but it also significantly speeds up the training. Our proposed method, called MIRO, achieves state-of-the-art performances on the GenEval compositional benchmark and user-preference scores (PickAScore, ImageReward, HPSv2).

View on arXivView PDF

Code Implementations(1)

nicolas-dufour/miroCommunity69%

Code for MIRO: MultI-Reward cOnditioned pretraining improves T2I quality and efficiency

401Oct 31, 202511 months ago

Similar Papers

DiT-Reward: Generative Representations for Text-to-Image Reward Modeling

86% match2026

Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation

84% match2026

Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation

84% match2024

ParetoSlider: Diffusion Models Post-Training for Continuous Reward Control

84% match2026

Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation

83% match2024

Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning

83% match1 repo2492025

Cite this paper

@article{dufour2025miro,
  title  = {MIRO: MultI-Reward cOnditioned pretraining improves T2I quality and efficiency},
  author = {Nicolas Dufour and Lucas Degeorge and Arijit Ghosh and Vicky Kalogeiton and David Picard},
  year   = {2025},
  eprint = {2510.25897},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2510.25897v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2510.25897.svg)](https://codeofpaper.com/paper/2510.25897)

Discussion