Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics

Antonin Raffin, Ashley Hill, René Traoré, Timothée Lesort, Natalia Díaz-Rodríguez +1 more
1/24/2019
cs.LGcs.ROstat.ML

Official code: araffin/robotics-rl-srl (655 stars, MIT, PyTorch, last commit 2021-04-05).

Abstract

Scaling end-to-end reinforcement learning to control real robots from vision presents a series of challenges, in particular in terms of sample efficiency. Against end-to-end learning, state representation learning can help learn a compact, efficient and relevant representation of states that speeds up policy learning, reducing the number of samples needed, and that is easier to interpret. We evaluate several state representation learning methods on goal based robotics tasks and propose a new unsupervised model that stacks representations and combines strengths of several of these approaches. This method encodes all the relevant features, performs on par or better than end-to-end learning with better sample efficiency, and is robust to hyper-parameters change.

View on arXivView PDF

Code Implementations(1)

araffin/robotics-rl-srlOfficial100%
PyTorchMIT

S-RL Toolbox: Reinforcement Learning (RL) and State Representation Learning (SRL) for Robotics

65592Shell, PythonJan 18, 20185 years agoMIT
armbaselinesbaxter-robotgymkuka+6 more

Similar Papers

State Representations as Incentives for Reinforcement Learning Agents: A Sim2Real Analysis on Robotic Grasping

87% match1 repo42023

S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning

85% match1 repo6552018

EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data

85% match1 repo192024

Visual Reinforcement Learning with Imagined Goals

85% match2 repos2.9k2018

A Survey of State Representation Learning for Deep Reinforcement Learning

84% match2025

Improving Reinforcement Learning Efficiency with Auxiliary Tasks in Non-Visual Environments: A Comparison

83% match2023

Cite this paper

@article{raffin2019decoupling,
  title  = {Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics},
  author = {Antonin Raffin and Ashley Hill and René Traoré and Timothée Lesort and Natalia Díaz-Rodríguez and David Filliat},
  year   = {2019},
  eprint = {1901.08651},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1901.08651}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1901.08651.svg)](https://codeofpaper.com/paper/1901.08651)

Discussion