Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity

Samin Yeasar Arnob, Scott Fujimoto, Doina Precup
6/20/2025
cs.LGcs.AI

No official code found. Best implementation: SaminYeasar/sparse_reg (0 stars, last commit 2025-06-26).

Abstract

In this paper, we investigate the use of small datasets in the context of offline reinforcement learning (RL). While many common offline RL benchmarks employ datasets with over a million data points, many offline RL applications rely on considerably smaller datasets. We show that offline RL algorithms can overfit on small datasets, resulting in poor performance. To address this challenge, we introduce "Sparse-Reg": a regularization technique based on sparsity to mitigate overfitting in offline reinforcement learning, enabling effective learning in limited data settings and outperforming state-of-the-art baselines in continuous control.

View on arXivView PDF

Code Implementations(1)

SaminYeasar/sparse_regCommunity100%

Implementation of "Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity"

00PythonJun 26, 20251 years ago

Similar Papers

Sparsity-based Safety Conservatism for Constrained Offline Reinforcement Learning

87% match2024

Learning from Sparse Offline Datasets via Conservative Density Estimation

86% match1 repo32024

Efficient Reinforcement Learning by Guiding Generalist World Models with Non-Curated Data

85% match2025

On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond

85% match2024

On the Statistical Complexity for Offline and Low-Adaptive Reinforcement Learning with Structures

85% match2025

Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data

85% match1 repo1502024

Cite this paper

@article{arnob2025sparsereg,
  title  = {Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity},
  author = {Samin Yeasar Arnob and Scott Fujimoto and Doina Precup},
  year   = {2025},
  eprint = {2506.17155},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2506.17155v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2506.17155.svg)](https://codeofpaper.com/paper/2506.17155)

Discussion