Spacetime $E(n)$-Transformer: Equivariant Attention for Spatio-temporal Graphs

Sergio G. Charles
8/12/2024
cs.LGcs.AI

No official code found. Best implementation: sergiogcharles/set (0 stars, last commit 2024-10-02).

Abstract

We introduce an $E(n)$-equivariant Transformer architecture for spatio-temporal graph data. By imposing rotation, translation, and permutation equivariance inductive biases in both space and time, we show that the Spacetime $E(n)$-Transformer (SET) outperforms purely spatial and temporal models without symmetry-preserving properties. We benchmark SET against said models on the charged $N$-body problem, a simple physical system with complex dynamics. While existing spatio-temporal graph neural networks focus on sequential modeling, we empirically demonstrate that leveraging underlying domain symmetries yields considerable improvements for modeling dynamical systems on graphs.

View on arXivView PDF

Code Implementations(1)

sergiogcharles/setCommunity100%

Spacetime E(n)-Transformer: Equivariant Attention for Spatio-temporal Graphs

00Shell, PythonOct 2, 20242 years ago

Similar Papers

Spatial Attention Kinetic Networks with E(n)-Equivariance

83% match1 repo92023

Equivariant Spatio-Temporal Attentive Graph Networks to Simulate Physical Dynamics

82% match1 repo282024

Learning with Category-Equivariant Architectures for Human Activity Recognition

81% match2025

Spatially-informed transformers: Injecting geostatistical covariance biases into self-attention for spatio-temporal forecasting

80% match2025

SiT: Symmetry-Invariant Transformers for Generalisation in Reinforcement Learning

79% match1 repo2024

Neural Spacetimes for DAG Representation Learning

79% match1 repo142024

Cite this paper

@article{charles2024spacetime,
  title  = {Spacetime $E(n)$-Transformer: Equivariant Attention for Spatio-temporal Graphs},
  author = {Sergio G. Charles},
  year   = {2024},
  eprint = {2408.06039},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2408.06039v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2408.06039.svg)](https://codeofpaper.com/paper/2408.06039)

Discussion