Unsupervised State Representation Learning in Atari

Ankesh Anand, Evan Racah, Sherjil Ozair, Yoshua Bengio, Marc-Alexandre Côté +1 more
6/19/2019
cs.LGstat.ML

Official code: mila-iqia/atari-representation-learning (259 stars, MIT, last commit 2023-11-02).

Abstract

State representation learning, or the ability to capture latent generative factors of an environment, is crucial for building intelligent agents that can perform a wide variety of tasks. Learning such representations without supervision from rewards is a challenging open problem. We introduce a method that learns state representations by maximizing mutual information across spatially and temporally distinct features of a neural encoder of the observations. We also introduce a new benchmark based on Atari 2600 games where we evaluate representations based on how well they capture the ground truth state variables. We believe this new framework for evaluating representation learning models will be crucial for future representation learning research. Finally, we compare our technique with other state-of-the-art generative and contrastive representation learning methods. The code associated with this work is available at https://github.com/mila-iqia/atari-representation-learning

View on arXivView PDF

Code Implementations(1)

mila-iqia/atari-representation-learningOfficial100%
MIT

Code for "Unsupervised State Representation Learning in Atari"

25952Python, Jupyter NotebookJan 23, 20192 years agoMIT

Similar Papers

Unsupervised Representation Learning in Partially Observable Atari Games

88% match2023

S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning

85% match1 repo6552018

Learning deep representations by mutual information estimation and maximization

85% match1 repo8102018

Value Explicit Pretraining for Learning Transferable Representations

83% match2023

MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments

83% match1 repo3352019

A Survey of State Representation Learning for Deep Reinforcement Learning

83% match2025

Cite this paper

@article{anand2019unsupervised,
  title  = {Unsupervised State Representation Learning in Atari},
  author = {Ankesh Anand and Evan Racah and Sherjil Ozair and Yoshua Bengio and Marc-Alexandre Côté and R Devon Hjelm},
  year   = {2019},
  eprint = {1906.08226},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1906.08226}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1906.08226.svg)](https://codeofpaper.com/paper/1906.08226)

Discussion