Transformers as Policies for Variable Action Environments

Niklas Zwingenberger
1/9/2023
cs.AIcs.LG

Official code: NiklasZ/transformers-for-variable-action-envs (9 stars, MIT, last commit 2026-04-17).

Abstract

In this project we demonstrate the effectiveness of the transformer encoder as a viable architecture for policies in variable action environments. Using it, we train an agent using Proximal Policy Optimisation (PPO) on multiple maps against scripted opponents in the Gym-$μ$RTS environment. The final agent is able to achieve a higher return using half the computational resources of the next-best RL agent, which used the GridNet architecture. The source code and pre-trained models are available here: https://github.com/NiklasZ/transformers-for-variable-action-envs

View on arXivView PDF

Code Implementations(1)

NiklasZ/transformers-for-variable-action-envsOfficial100%
MIT
91Shell, PythonMay 18, 20226 months agoMIT

Similar Papers

Utilizing Evolution Strategies to Train Transformers in Reinforcement Learning

83% match1 repo32025

Do Transformer World Models Give Better Policy Gradients?

80% match2024

Transformers in Reinforcement Learning: A Survey

80% match1 repo542023

Do We Need Transformers to Play FPS Video Games?

79% match2025

An Optimal Control Approach To Transformer Training

78% match2026

Hyper-Decision Transformer for Efficient Online Policy Adaptation

78% match2023

Cite this paper

@article{zwingenberger2023transformers,
  title  = {Transformers as Policies for Variable Action Environments},
  author = {Niklas Zwingenberger},
  year   = {2023},
  eprint = {2301.03679},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.03679v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.03679.svg)](https://codeofpaper.com/paper/2301.03679)

Discussion