RIME: Robust Preference-based Reinforcement Learning with Noisy Preferences

Jie Cheng, Gang Xiong, Xingyuan Dai, Q. Miao, Yisheng Lv +1 more
2/12/2026

No official code found. Best implementation: CJReinforce/RIME_ICML2024 (35 stars, MIT, last commit 2024-10-15).

Abstract

Preference-based Reinforcement Learning (PbRL) circumvents the need for reward engineering by harnessing human preferences as the reward signal. However, current PbRL methods excessively depend on high-quality feedback from domain experts, which results in a lack of robustness. In this paper, we present RIME, a robust PbRL algorithm for effective reward learning from noisy preferences. Our method utilizes a sample selection-based discriminator to dynamically filter out noise and ensure robust training. To counteract the cumulative error stemming from incorrect selection, we suggest a warm start for the reward model, which additionally bridges the performance gap during the transition from pre-training to online training in PbRL. Our experiments on robotic manipulation and locomotion tasks demonstrate that RIME significantly enhances the robustness of the state-of-the-art PbRL method. Code is available at https://github.com/CJReinforce/RIME_ICML2024.

View on arXivView PDF

Code Implementations(1)

CJReinforce/RIME_ICML2024Community60%
MIT

Official code for ICML 2024 paper, "RIME: Robust Preference-based Reinforcement Learning with Noisy Preferences" (ICML 2024 Spotlight)

354Shell, PythonApr 4, 20242 years agoMIT
artificial-intelligencedeep-learninglocomotionmanipulationpreference-learning+3 more

Similar Papers

RIME: Robust Preference-based Reinforcement Learning with Noisy Preferences

100% match1 repo352024

SENIOR: Efficient Query Selection and Preference-Guided Exploration in Preference-based Reinforcement Learning

87% match2025

S-EPOA: Overcoming the Indistinguishability of Segments with Skill-Driven Preference-Based Reinforcement Learning

86% match1 repo12024

Robust Reinforcement Learning from Corrupted Human Feedback

86% match2024

Boosting Robustness in Preference-Based Reinforcement Learning with Dynamic Sparsity

85% match2024

Learning Reward for Robot Skills Using Large Language Models via Self-Alignment

85% match1 repo192026

Cite this paper

@article{cheng2026rime,
  title  = {RIME: Robust Preference-based Reinforcement Learning with Noisy Preferences},
  author = {Jie Cheng and Gang Xiong and Xingyuan Dai and Q. Miao and Yisheng Lv and Fei-Yue Wang},
  year   = {2026},
  eprint = {2402.17257},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2402.17257},
  journal = {ICML 2024 2024}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.17257.svg)](https://codeofpaper.com/paper/2402.17257)

Discussion