JointMotion: Joint Self-Supervision for Joint Motion Prediction

Royden Wagner, Omer Sahin Tas, Marvin Klemp, Carlos Fernandez
3/8/2024
cs.CVcs.RO

Official code: KIT-MRT/future-motion (101 stars, last commit 2026-03-21).

Abstract

We present JointMotion, a self-supervised pre-training method for joint motion prediction in self-driving vehicles. Our method jointly optimizes a scene-level objective connecting motion and environments, and an instance-level objective to refine learned representations. Scene-level representations are learned via non-contrastive similarity learning of past motion sequences and environment context. At the instance level, we use masked autoencoding to refine multimodal polyline representations. We complement this with an adaptive pre-training decoder that enables JointMotion to generalize across different environment representations, fusion mechanisms, and dataset characteristics. Notably, our method reduces the joint final displacement error of Wayformer, HPTR, and Scene Transformer models by 3\%, 8\%, and 12\%, respectively; and enables transfer learning between the Waymo Open Motion and the Argoverse 2 Motion Forecasting datasets. Code: https://github.com/kit-mrt/future-motion

View on arXivView PDF

Code Implementations(1)

KIT-MRT/future-motionOfficial100%

Common repo for our research on motion forecasting for self-driving cars

10110Shell, PythonMay 22, 20247 months ago

Similar Papers

SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Prediction

90% match1 repo422026

Multi-Transmotion: Pre-trained Model for Human Motion Prediction

87% match1 repo382024

MoReFun: Past-Movement Guided Motion Representation Learning for Future Motion Prediction and Understanding

87% match1 repo2024

Self-Supervised Representation Learning with Joint Embedding Predictive Architecture for Automotive LiDAR Object Detection

85% match1 repo472025

Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders

85% match2 repos2222026

Weakly and Self-Supervised Class-Agnostic Motion Prediction for Autonomous Driving

85% match2025

Cite this paper

@article{wagner2024jointmotion,
  title  = {JointMotion: Joint Self-Supervision for Joint Motion Prediction},
  author = {Royden Wagner and Omer Sahin Tas and Marvin Klemp and Carlos Fernandez},
  year   = {2024},
  eprint = {2403.05489},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.05489v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.05489.svg)](https://codeofpaper.com/paper/2403.05489)

Discussion