On the Benefits of 3D Pose and Tracking for Human Action Recognition

Jathushan Rajasegaran, G. Pavlakos, Angjoo Kanazawa, Christoph Feichtenhofer, J. Malik
2/14/2026

No official code found. Best implementation: brjathu/LART (287 stars, last commit 2024-01-19).

Abstract

In this work we study the benefits of using tracking and 3D poses for action recognition. To achieve this, we take the Lagrangian view on analysing actions over a trajectory of human motion rather than at a fixed point in space. Taking this stand allows us to use the tracklets of people to predict their actions. In this spirit, first we show the benefits of using 3D pose to infer actions, and study person-person in-teractions. Subsequently, we propose a Lagrangian Action Recognition model by fusing 3D pose and contextualized appearance over tracklets. To this end, our method achieves state-of-the-art performance on the AVA v2.2 dataset on both pose only settings and on standard benchmark settings. When reasoning about the action using only pose cues, our pose model achieves +10.0 mAP gain over the corresponding state-of-the-art while our fused model has a gain of +2.8 mAP over the best state-of-the-art model. Code and results are available at: https://brjathu.github.io/LART

View on arXivView PDF

Code Implementations(1)

brjathu/LARTCommunity100%

Code repository for the paper "On the Benefits of 3D Pose and Tracking for Human Action Recognition", (CVPR 2023)

28734Shell, PythonJun 19, 20232 years ago

Similar Papers

DreamPose3D: Hallucinative Diffusion with Prompt Learning for 3D Human Pose Estimation

85% match2025

TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments

84% match2 repos1.5k2026

Collaboratively Self-supervised Video Representation Learning for Action Recognition

84% match2024

Just Add π! Pose Induced Video Transformers for Understanding Activities of Daily Living

84% match1 repo322026

LangPose: Language-Aligned Motion for Robust 3D Human Pose Estimation

83% match2024

Grounding Foundational Vision Models with 3D Human Poses for Robust Action Recognition

83% match2025

Cite this paper

@article{rajasegaran2026benefits,
  title  = {On the Benefits of 3D Pose and Tracking for Human Action Recognition},
  author = {Jathushan Rajasegaran and G. Pavlakos and Angjoo Kanazawa and Christoph Feichtenhofer and J. Malik},
  year   = {2026},
  eprint = {2304.01199},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/pdf/2304.01199},
  journal = {CVPR 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.01199.svg)](https://codeofpaper.com/paper/2304.01199)

Discussion