Learning Rewards to Optimize Global Performance Metrics in Deep Reinforcement Learning

Junqi Qian, Paul Weng, Chenmien Tan
3/16/2023
cs.LG

No confident code implementation found for this paper.

Abstract

When applying reinforcement learning (RL) to a new problem, reward engineering is a necessary, but often difficult and error-prone task a system designer has to face. To avoid this step, we propose LR4GPM, a novel (deep) RL method that can optimize a global performance metric, which is supposed to be available as part of the problem description. LR4GPM alternates between two phases: (1) learning a (possibly vector) reward function used to fit the performance metric, and (2) training a policy to optimize an approximation of this performance metric based on the learned rewards. Such RL training is not straightforward since both the reward function and the policy are trained using non-stationary data. To overcome this issue, we propose several training tricks. We demonstrate the efficiency of LR4GPM on several domains. Notably, LR4GPM outperforms the winner of a recent autonomous driving competition organized at DAI'2020.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications

86% match2024

Reward Models in Deep Reinforcement Learning: A Survey

86% match2025

Leveraging LLMs for reward function design in reinforcement learning control tasks

86% match2025

Performance Optimization of Ratings-Based Reinforcement Learning

86% match2025

Is Exploration or Optimization the Problem for Deep Reinforcement Learning?

85% match2025

On the Importance of Reward Design in Reinforcement Learning-based Dynamic Algorithm Configuration: A Case Study on OneMax with (1+(λ,λ))-GA

85% match2025

Cite this paper

@article{qian2023learning,
  title  = {Learning Rewards to Optimize Global Performance Metrics in Deep Reinforcement Learning},
  author = {Junqi Qian and Paul Weng and Chenmien Tan},
  year   = {2023},
  eprint = {2303.09027},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.09027v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.09027.svg)](https://codeofpaper.com/paper/2303.09027)

Discussion