Practical Probabilistic Model-based Deep Reinforcement Learning by Integrating Dropout Uncertainty and Trajectory Sampling

Wenjun Huang, Yunduan Cui, Huiyun Li, Xinyu Wu
9/20/2023
eess.SYcs.AIcs.LG

Official code: mrjun123/DPETS (16 stars, last commit 2024-10-22).

Abstract

This paper addresses the prediction stability, prediction accuracy and control capability of the current probabilistic model-based reinforcement learning (MBRL) built on neural networks. A novel approach dropout-based probabilistic ensembles with trajectory sampling (DPETS) is proposed where the system uncertainty is stably predicted by combining the Monte-Carlo dropout and trajectory sampling in one framework. Its loss function is designed to correct the fitting error of neural networks for more accurate prediction of probabilistic models. The state propagation in its policy is extended to filter the aleatoric uncertainty for superior control capability. Evaluated by several Mujoco benchmark control tasks under additional disturbances and one practical robot arm manipulation task, DPETS outperforms related MBRL approaches in both average return and convergence velocity while achieving superior performance than well-known model-free baselines with significant sample efficiency. The open source code of DPETS is available at https://github.com/mrjun123/DPETS.

View on arXivView PDF

Code Implementations(1)

mrjun123/DPETSOfficial100%

Code for the paper "Practical Probabilistic Model-based Deep Reinforcement Learning by Integrating Dropout Uncertainty and Trajectory Sampling"

165PythonMay 15, 20231 years ago

Similar Papers

Dropout MPC: An Ensemble Neural MPC Approach for Systems with Learned Dynamics

88% match2024

Deep Gaussian Covariance Network with Trajectory Sampling for Data-Efficient Policy Search

87% match2024

Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learning

84% match2023

DADEE: Well-calibrated uncertainty quantification in neural networks for barriers-based robot safety

83% match2024

Fast Value Tracking for Deep Reinforcement Learning

82% match2024

Environment Transformer and Policy Optimization for Model-Based Offline Reinforcement Learning

82% match2023

Cite this paper

@article{huang2023practical,
  title  = {Practical Probabilistic Model-based Deep Reinforcement Learning by Integrating Dropout Uncertainty and Trajectory Sampling},
  author = {Wenjun Huang and Yunduan Cui and Huiyun Li and Xinyu Wu},
  year   = {2023},
  eprint = {2309.11089},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.11089v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.11089.svg)](https://codeofpaper.com/paper/2309.11089)

Discussion