Risk-Aware Active Inverse Reinforcement Learning

Daniel S. Brown, Yuchen Cui, Scott Niekum
1/8/2019
cs.LGstat.ML

Official code: Pearl-UTexas/ActiveVaR (17 stars, last commit 2018-12-20).

Abstract

Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning. Existing work has explored a variety of active query strategies; however, to our knowledge, none of these strategies directly minimize the performance risk of the policy the robot is learning. Utilizing recent advances in performance bounds for inverse reinforcement learning, we propose a risk-aware active inverse reinforcement learning algorithm that focuses active queries on areas of the state space with the potential for large generalization error. We show that risk-aware active learning outperforms standard active IRL approaches on gridworld, simulated driving, and table setting tasks, while also providing a performance-based stopping criterion that allows a robot to know when it has received enough demonstrations to safely perform a task.

View on arXivView PDF

Code Implementations(1)

Pearl-UTexas/ActiveVaROfficial100%

Code for CORL'18 paper "Risk-Aware Active Inverse Reinforcement Learning"

170C++, PythonSep 28, 20187 years ago

Similar Papers

Toward Information Theoretic Active Inverse Reinforcement Learning

86% match2024

Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learning

85% match1 repo52017

PAC Apprenticeship Learning with Bayesian Active Inverse Reinforcement Learning

84% match2025

SPLASH! Sample-efficient Preference-based inverse reinforcement learning for Long-horizon Adversarial tasks from Suboptimal Hierarchical demonstrations

83% match2025

Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environments

82% match2023

Safe Deep Reinforcement Learning by Verifying Task-Level Properties

82% match2023

Cite this paper

@article{brown2019riskaware,
  title  = {Risk-Aware Active Inverse Reinforcement Learning},
  author = {Daniel S. Brown and Yuchen Cui and Scott Niekum},
  year   = {2019},
  eprint = {1901.02161},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1901.02161}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1901.02161.svg)](https://codeofpaper.com/paper/1901.02161)

Discussion