Dataset Distillation for Offline Reinforcement Learning

Jonathan Light, Yuanzhe Liu, Ziniu Hu
7/29/2024
cs.LGcs.AI

Official code: ggflow123/DDRL (9 stars, Apache-2.0, last commit 2024-08-02).

Abstract

Offline reinforcement learning often requires a quality dataset that we can train a policy on. However, in many situations, it is not possible to get such a dataset, nor is it easy to train a policy to perform well in the actual environment given the offline data. We propose using data distillation to train and distill a better dataset which can then be used for training a better policy model. We show that our method is able to synthesize a dataset where a model trained on it achieves similar performance to a model trained on the full dataset or a model trained using percentile behavioral cloning. Our project site is available at https://datasetdistillation4rl.github.io . We also provide our implementation at https://github.com/ggflow123/DDRL .

View on arXivView PDF

Code Implementations(1)

ggflow123/DDRLOfficial100%
Apache-2.0

Repository for Data Distillation for Offline Reinforcement Learning

90PythonJul 12, 20242 years agoApache-2.0

Similar Papers

Offline Behavior Distillation

87% match2026

Policy Regularization with Dataset Constraint for Offline Reinforcement Learning

86% match1 repo182023

Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data

86% match2023

Sample-Efficient Policy Constraint Offline Deep Reinforcement Learning based on Sample Filtering

86% match2025

Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model

85% match2025

Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets

85% match1 repo252026

Cite this paper

@article{light2024dataset,
  title  = {Dataset Distillation for Offline Reinforcement Learning},
  author = {Jonathan Light and Yuanzhe Liu and Ziniu Hu},
  year   = {2024},
  eprint = {2407.20299},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.20299v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.20299.svg)](https://codeofpaper.com/paper/2407.20299)

Discussion