Improving Environment Robustness of Deep Reinforcement Learning Approaches for Autonomous Racing Using Bayesian Optimization-based Curriculum Learning

Rohan Banerjee, Prishita Ray, Mark Campbell
12/16/2023
cs.ROcs.LG

Official code: PRISHIta123/Curriculum_RL_for_Driving (3 stars, MIT, last commit 2023-12-19).

Abstract

Deep reinforcement learning (RL) approaches have been broadly applied to a large number of robotics tasks, such as robot manipulation and autonomous driving. However, an open problem in deep RL is learning policies that are robust to variations in the environment, which is an important condition for such systems to be deployed into real-world, unstructured settings. Curriculum learning is one approach that has been applied to improve generalization performance in both supervised and reinforcement learning domains, but selecting the appropriate curriculum to achieve robustness can be a user-intensive process. In our work, we show that performing probabilistic inference of the underlying curriculum-reward function using Bayesian Optimization can be a promising technique for finding a robust curriculum. We demonstrate that a curriculum found with Bayesian optimization can outperform a vanilla deep RL agent and a hand-engineered curriculum in the domain of autonomous racing with obstacle avoidance. Our code is available at https://github.com/PRISHIta123/Curriculum_RL_for_Driving.

View on arXivView PDF

Code Implementations(1)

PRISHIta123/Curriculum_RL_for_DrivingOfficial100%
MIT

Curriculum Design through Bayesian Optimization to train RL agents for the purpose of autonomous racing in the OpenAI Gym CarRacing Environment

31Shell, PythonMay 12, 20232 years agoMIT
autonomous-drivingbayesian-optimizationcurriculum-learningreinforcement-learning

Similar Papers

Scaling Is All You Need: Autonomous Driving with JAX-Accelerated Reinforcement Learning

87% match2023

Stabilizing Unsupervised Environment Design with a Learned Adversary

87% match1 repo1412023

Learning Curricula in Open-Ended Worlds

87% match1 repo1412023

Learning Visual Tracking and Reaching with Deep Reinforcement Learning on a UR10e Robotic Arm

87% match1 repo42023

Efficient Exploration Using Extra Safety Budget in Constrained Policy Optimization

86% match1 repo82023

Probabilistic Curriculum Learning for Goal-Based Reinforcement Learning

86% match2025

Cite this paper

@article{banerjee2023improving,
  title  = {Improving Environment Robustness of Deep Reinforcement Learning Approaches for Autonomous Racing Using Bayesian Optimization-based Curriculum Learning},
  author = {Rohan Banerjee and Prishita Ray and Mark Campbell},
  year   = {2023},
  eprint = {2312.10557},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.10557v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.10557.svg)](https://codeofpaper.com/paper/2312.10557)

Discussion