Scalable Multi-Objective Reinforcement Learning with Fairness Guarantees using Lorenz Dominance

Dimitris Michailidis, Willem Röpke, Diederik M. Roijers, Sennay Ghebreab, Fernando P. Santos
11/27/2024
cs.LG

No official code found. Best implementation: dimichai/mo-tndp (11 stars, MIT, last commit 2026-02-27).

Abstract

Multi-Objective Reinforcement Learning (MORL) aims to learn a set of policies that optimize trade-offs between multiple, often conflicting objectives. MORL is computationally more complex than single-objective RL, particularly as the number of objectives increases. Additionally, when objectives involve the preferences of agents or groups, ensuring fairness is socially desirable. This paper introduces a principled algorithm that incorporates fairness into MORL while improving scalability to many-objective problems. We propose using Lorenz dominance to identify policies with equitable reward distributions and introduce λ-Lorenz dominance to enable flexible fairness preferences. We release a new, large-scale real-world transport planning environment and demonstrate that our method encourages the discovery of fair policies, showing improved scalability in two large cities (Xi'an and Amsterdam). Our methods outperform common multi-objective approaches, particularly in high-dimensional objective spaces.

View on arXivView PDF

Code Implementations(1)

dimichai/mo-tndpCommunity100%
MIT

Multi-Objective Transport Network Design Problem (MO-TNDP) gymnasium environment.

112PythonApr 11, 20237 months agoMIT
multi-objective-optimizationmulti-objective-reinforcement-learningmulti-objective-rlreinforcement-learningtransport-planning

Similar Papers

Preference-based Multi-Objective Reinforcement Learning

85% match2025

Fairness in Preference-based Reinforcement Learning

85% match2023

Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization

84% match2026

Group Fairness in Multi-Task Reinforcement Learning

83% match2025

Pareto Set Learning for Multi-Objective Reinforcement Learning

83% match2025

Interpretability by Design for Efficient Multi-Objective Reinforcement Learning

83% match2025

Cite this paper

@article{michailidis2024scalable,
  title  = {Scalable Multi-Objective Reinforcement Learning with Fairness Guarantees using Lorenz Dominance},
  author = {Dimitris Michailidis and Willem Röpke and Diederik M. Roijers and Sennay Ghebreab and Fernando P. Santos},
  year   = {2024},
  eprint = {2411.18195},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2411.18195v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2411.18195.svg)](https://codeofpaper.com/paper/2411.18195)

Discussion