A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning

Abdulaziz Almuzairee, Nicklas Hansen, Henrik I. Christensen
5/27/2024
cs.LGcs.CVcs.RO

No official code found. Best implementation: aalmuzairee/dmcgb2 (22 stars, MIT, last commit 2025-07-21). 1 more below.

Abstract

Q-learning algorithms are appealing for real-world applications due to their data-efficiency, but they are very prone to overfitting and training instabilities when trained from visual observations. Prior work, namely SVEA, finds that selective application of data augmentation can improve the visual generalization of RL agents without destabilizing training. We revisit its recipe for data augmentation, and find an assumption that limits its effectiveness to augmentations of a photometric nature. Addressing these limitations, we propose a generalized recipe, SADA, that works with wider varieties of augmentations. We benchmark its effectiveness on DMC-GB2 - our proposed extension of the popular DMControl Generalization Benchmark - as well as tasks from Meta-World and the Distracting Control Suite, and find that our method, SADA, greatly improves training stability and generalization of RL agents across a diverse set of augmentations. For visualizations, code and benchmark: see https://aalmuzairee.github.io/SADA/

View on arXivView PDF

Code Implementations(2)

aalmuzairee/dmcgb2Community100%
MIT

Official release of the DMControl Generalization Benchmark 2 (DMC-GB2)

224Shell, PythonMay 28, 20241 years agoMIT
data-augumentationroboticsvisual-reinforcement-learning
aalmuzairee/SADACommunity66%

Website for "A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning"

00CSS, HTMLJun 3, 20246 months ago

Similar Papers

Revisiting Data Augmentation in Deep Reinforcement Learning

87% match1 repo62024

Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning

87% match2023

Synthetic Data is Sufficient for Zero-Shot Visual Generalization from Offline Data

84% match2025

Zero-Shot Generalization of Vision-Based RL Without Data Augmentation

84% match2024

Salience-Invariant Consistent Policy Learning for Generalization in Visual Reinforcement Learning

83% match2025

DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors

83% match1 repo352024

Cite this paper

@article{almuzairee2024recipe,
  title  = {A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning},
  author = {Abdulaziz Almuzairee and Nicklas Hansen and Henrik I. Christensen},
  year   = {2024},
  eprint = {2405.17416},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.17416v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.17416.svg)](https://codeofpaper.com/paper/2405.17416)

Discussion