Swapped goal-conditioned offline reinforcement learning

Wenyan Yang, Huiling Wang, Dingding Cai, Joni Pajarinen, Joni-Kristen Kämäräinen
2/17/2023
cs.LGcs.AI

No official code found. Best implementation: JasonMa2016/GoFAR (38 stars, MIT, last commit 2023-10-19).

Abstract

Offline goal-conditioned reinforcement learning (GCRL) can be challenging due to overfitting to the given dataset. To generalize agents' skills outside the given dataset, we propose a goal-swapping procedure that generates additional trajectories. To alleviate the problem of noise and extrapolation errors, we present a general offline reinforcement learning method called deterministic Q-advantage policy gradient (DQAPG). In the experiments, DQAPG outperforms state-of-the-art goal-conditioned offline RL methods in a wide range of benchmark tasks, and goal-swapping further improves the test results. It is noteworthy, that the proposed method obtains good performance on the challenging dexterous in-hand manipulation tasks for which the prior methods failed.

View on arXivView PDF

Code Implementations(1)

JasonMa2016/GoFARCommunity100%
MIT

Official repository for Paper "Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression" (NeurIPS 2022)

387PythonMay 29, 20223 years agoMIT

Similar Papers

Prioritized offline Goal-swapping Experience Replay

90% match2023

Goal-conditioned Offline Reinforcement Learning through State Space Partitioning

88% match2023

Score Models for Offline Goal-Conditioned Reinforcement Learning

87% match1 repo72026

Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning

86% match2023

Test-time Offline Reinforcement Learning on Goal-related Experience

86% match1 repo72025

Flattening Hierarchies with Policy Bootstrapping

86% match2025

Cite this paper

@article{yang2023swapped,
  title  = {Swapped goal-conditioned offline reinforcement learning},
  author = {Wenyan Yang and Huiling Wang and Dingding Cai and Joni Pajarinen and Joni-Kristen Kämäräinen},
  year   = {2023},
  eprint = {2302.08865},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.08865v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.08865.svg)](https://codeofpaper.com/paper/2302.08865)

Discussion