Value Prediction Network

Junhyuk Oh, Satinder Singh, Honglak Lee
7/11/2017
cs.AIcs.LG

Official code: junhyukoh/value-prediction-network (166 stars, last commit 2018-01-12).

Abstract

This paper proposes a novel deep reinforcement learning (RL) architecture, called Value Prediction Network (VPN), which integrates model-free and model-based RL methods into a single neural network. In contrast to typical model-based RL methods, VPN learns a dynamics model whose abstract states are trained to make option-conditional predictions of future values (discounted sum of rewards) rather than of future observations. Our experimental results show that VPN has several advantages over both model-free and model-based baselines in a stochastic environment where careful planning is required but building an accurate observation-prediction model is difficult. Furthermore, VPN outperforms Deep Q-Network (DQN) on several Atari games even with short-lookahead planning, demonstrating its potential as a new way of learning a good state representation.

View on arXivView PDF

Code Implementations(1)

junhyukoh/value-prediction-networkOfficial100%

NIPS 2017 Value Prediction Network

16640Shell, PythonOct 14, 20178 years ago

Similar Papers

Model-Based Reinforcement Learning for Atari

86% match1 repo17.5k2019

Neural ODE and SDE Models for Adaptation and Planning in Model-Based Reinforcement Learning

86% match1 repo62026

Model-Ensemble Trust-Region Policy Optimization

85% match1 repo912018

Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks

85% match2 repos1512016

StarCraft II: A New Challenge for Reinforcement Learning

85% match1 repo8.3k2017

Hieros: Hierarchical Imagination on Structured State Space Sequence World Models

85% match1 repo232023

Cite this paper

@article{oh2017value,
  title  = {Value Prediction Network},
  author = {Junhyuk Oh and Satinder Singh and Honglak Lee},
  year   = {2017},
  eprint = {1707.03497},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1707.03497}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1707.03497.svg)](https://codeofpaper.com/paper/1707.03497)

Discussion