PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning

Hangyu Mao, Rui Zhao, Ziyue Li, Zhiwei Xu, Hao Chen +5 more
12/26/2023
cs.LGcs.AIcs.RO

Official code: maohangyu/PDiT (10 stars, last commit 2023-12-27).

Abstract

Designing better deep networks and better reinforcement learning (RL) algorithms are both important for deep RL. This work studies the former. Specifically, the Perception and Decision-making Interleaving Transformer (PDiT) network is proposed, which cascades two Transformers in a very natural way: the perceiving one focuses on \emph{the environmental perception} by processing the observation at the patch level, whereas the deciding one pays attention to \emph{the decision-making} by conditioning on the history of the desired returns, the perceiver's outputs, and the actions. Such a network design is generally applicable to a lot of deep RL settings, e.g., both the online and offline RL algorithms under environments with either image observations, proprioception observations, or hybrid image-language observations. Extensive experiments show that PDiT can not only achieve superior performance than strong baselines in different settings but also extract explainable feature representations. Our code is available at \url{https://github.com/maohangyu/PDiT}.

View on arXivView PDF

Code Implementations(1)

maohangyu/PDiTOfficial100%

PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning. AAMAS 2024 (full paper with oral presentation).

101Dec 22, 20232 years ago

Similar Papers

D2RL: Deep Dense Architectures in Reinforcement Learning

87% match1 repo402020

MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learning

86% match1 repo252023

Intrinsically Motivated Learning of Causal World Models

85% match2022

Pittsburgh Learning Classifier Systems for Explainable Reinforcement Learning: Comparing with XCS

85% match2023

Deep Reinforcement Learning

84% match4 repos10.9k2018

GAN Based Top-Down View Synthesis in Reinforcement Learning Environments

84% match2024

Cite this paper

@article{mao2023pdit,
  title  = {PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning},
  author = {Hangyu Mao and Rui Zhao and Ziyue Li and Zhiwei Xu and Hao Chen and Yiqun Chen and Bin Zhang and Zhen Xiao and Junge Zhang and Jiangjin Yin},
  year   = {2023},
  eprint = {2312.15863},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.15863v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.15863.svg)](https://codeofpaper.com/paper/2312.15863)

Discussion