Model-Based Active Exploration

Pranav Shyam, Wojciech Jaśkowski, Faustino Gomez
10/29/2018
cs.LGcs.AIcs.IT

Official code: nnaisense/MAX (81 stars, last commit 2019-07-23).

Abstract

Efficient exploration is an unsolved problem in Reinforcement Learning which is usually addressed by reactively rewarding the agent for fortuitously encountering novel situations. This paper introduces an efficient active exploration algorithm, Model-Based Active eXploration (MAX), which uses an ensemble of forward models to plan to observe novel events. This is carried out by optimizing agent behaviour with respect to a measure of novelty derived from the Bayesian perspective of exploration, which is estimated using the disagreement between the futures predicted by the ensemble members. We show empirically that in semi-random discrete environments where directed exploration is critical to make progress, MAX is at least an order of magnitude more efficient than strong baselines. MAX scales to high-dimensional continuous environments where it builds task-agnostic models that can be used for any downstream task.

View on arXivView PDF

Code Implementations(1)

nnaisense/MAXOfficial100%

Code for reproducing experiments in Model-Based Active Exploration, ICML 2019

8118PythonMay 13, 20197 years ago
deep-reinforcement-learningexplorationmodel-based-rlreinforcement-learning

Similar Papers

Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration

86% match2026

Active Exploration for Inverse Reinforcement Learning

85% match1 repo142022

Active Neural Topological Mapping for Multi-Agent Exploration

85% match2023

Beyond Optimism: Exploration With Partially Observable Rewards

84% match1 repo52024

Counterfactual Conditional Likelihood Rewards for Multiagent Exploration

84% match2026

Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning

84% match2024

Cite this paper

@article{shyam2018modelbased,
  title  = {Model-Based Active Exploration},
  author = {Pranav Shyam and Wojciech Jaśkowski and Faustino Gomez},
  year   = {2018},
  eprint = {1810.12162},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1810.12162}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1810.12162.svg)](https://codeofpaper.com/paper/1810.12162)

Discussion