Evolving Constrained Reinforcement Learning Policy

Chengpeng Hu, Jiyuan Pei, Jialin Liu, Xin Yao
4/19/2023
cs.NEcs.AIcs.LG

No official code found. Best implementation: HcPlu/Evolutionary-Constrained-Reinforcement-Learning (4 stars, last commit 2023-04-10).

Abstract

Evolutionary algorithms have been used to evolve a population of actors to generate diverse experiences for training reinforcement learning agents, which helps to tackle the temporal credit assignment problem and improves the exploration efficiency. However, when adapting this approach to address constrained problems, balancing the trade-off between the reward and constraint violation is hard. In this paper, we propose a novel evolutionary constrained reinforcement learning (ECRL) algorithm, which adaptively balances the reward and constraint violation with stochastic ranking, and at the same time, restricts the policy's behaviour by maintaining a set of Lagrange relaxation coefficients with a constraint buffer. Extensive experiments on robotic control benchmarks show that our ECRL achieves outstanding performance compared to state-of-the-art algorithms. Ablation analysis shows the benefits of introducing stochastic ranking and constraint buffer.

View on arXivView PDF

Code Implementations(1)

HcPlu/Evolutionary-Constrained-Reinforcement-LearningCommunity100%

The implement of Evolutionary Constrained Reinforcement Learning

42PythonDec 8, 20223 years ago

Similar Papers

Constrained Reinforcement Learning with Smoothed Log Barrier Function

85% match1 repo92024

CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learning

84% match2024

Incentivizing Safer Actions in Policy Optimization for Constrained Reinforcement Learning

84% match2025

Probabilistic Satisfaction of Temporal Logic Constraints in Reinforcement Learning via Adaptive Policy-Switching

84% match2024

Reduced Policy Optimization for Continuous Control with Hard Constraints

83% match1 repo462026

Constraints as Rewards: Reinforcement Learning for Robots without Reward Functions

83% match2025

Cite this paper

@article{hu2023evolving,
  title  = {Evolving Constrained Reinforcement Learning Policy},
  author = {Chengpeng Hu and Jiyuan Pei and Jialin Liu and Xin Yao},
  year   = {2023},
  eprint = {2304.09869},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2304.09869v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.09869.svg)](https://codeofpaper.com/paper/2304.09869)

Discussion