Estimating Risk and Uncertainty in Deep Reinforcement Learning

William R. Clements, Bastien Van Delft, Benoît-Marie Robaglia, Reda Bahi Slaoui, Sébastien Toth
5/23/2019
cs.LGcs.AIstat.ML

Official code: IndustAI/risk-and-uncertainty (11 stars, MIT, last commit 2023-10-03).

Abstract

Reinforcement learning agents are faced with two types of uncertainty. Epistemic uncertainty stems from limited data and is useful for exploration, whereas aleatoric uncertainty arises from stochastic environments and must be accounted for in risk-sensitive applications. We highlight the challenges involved in simultaneously estimating both of them, and propose a framework for disentangling and estimating these uncertainties on learned Q-values. We derive unbiased estimators of these uncertainties and introduce an uncertainty-aware DQN algorithm, which we show exhibits safe learning behavior and outperforms other DQN variants on the MinAtar testbed.

View on arXivView PDF

Code Implementations(1)

IndustAI/risk-and-uncertaintyOfficial100%
MIT

Code associated with our paper "Estimating Risk and Uncertainty in Reinforcement Learning"

114Python, Jupyter NotebookFeb 14, 20203 years agoMIT

Similar Papers

Training of Neural Networks with Uncertain Data: A Mixture of Experts Approach

86% match2023

What Uncertainties Do We Need for Dynamical Systems?

86% match2026

A unified uncertainty-aware exploration: Combining epistemic and aleatory uncertainty

85% match2024

Learning to Forecast Aleatoric and Epistemic Uncertainties over Long Horizon Trajectories

84% match2023

Explainability through uncertainty: Trustworthy decision-making with neural networks

84% match2024

Subjective Risk Decomposition: A New View for Uncertainty Quantification

84% match2026

Cite this paper

@article{clements2019estimating,
  title  = {Estimating Risk and Uncertainty in Deep Reinforcement Learning},
  author = {William R. Clements and Bastien Van Delft and Benoît-Marie Robaglia and Reda Bahi Slaoui and Sébastien Toth},
  year   = {2019},
  eprint = {1905.09638},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1905.09638}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1905.09638.svg)](https://codeofpaper.com/paper/1905.09638)

Discussion