Local Spherical Harmonics Improve Skeleton-Based Hand Action Recognition

Katharina Prasse, Steffen Jung, Yuxuan Zhou, Margret Keuper
8/21/2023
cs.CV

Official code: KathPra/LSHR_LSHT (9 stars, last commit 2024-04-05).

Abstract

Hand action recognition is essential. Communication, human-robot interactions, and gesture control are dependent on it. Skeleton-based action recognition traditionally includes hands, which belong to the classes which remain challenging to correctly recognize to date. We propose a method specifically designed for hand action recognition which uses relative angular embeddings and local Spherical Harmonics to create novel hand representations. The use of Spherical Harmonics creates rotation-invariant representations which make hand action recognition even more robust against inter-subject differences and viewpoint changes. We conduct extensive experiments on the hand joints in the First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations, and on the NTU RGB+D 120 dataset, demonstrating the benefit of using Local Spherical Harmonics Representations. Our code is available at https://github.com/KathPra/LSHR_LSHT.

View on arXivView PDF

Code Implementations(1)

KathPra/LSHR_LSHTOfficial100%
90PythonAug 16, 20232 years agoNOASSERTION

Similar Papers

CaSAR: Contact-aware Skeletal Action Recognition

87% match1 repo2023

Human Action Recognition in Egocentric Perspective Using 2D Object and Hands Pose

84% match2023

Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformation

83% match1 repo92024

CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-World

82% match1 repo552025

OmniHands: Towards Robust 4D Hand Mesh Recovery via A Versatile Transformer

82% match2024

In My Perspective, In My Hands: Accurate Egocentric 2D Hand Pose and Action Recognition

82% match1 repo112024

Cite this paper

@article{prasse2023local,
  title  = {Local Spherical Harmonics Improve Skeleton-Based Hand Action Recognition},
  author = {Katharina Prasse and Steffen Jung and Yuxuan Zhou and Margret Keuper},
  year   = {2023},
  eprint = {2308.10557},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2308.10557v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2308.10557.svg)](https://codeofpaper.com/paper/2308.10557)

Discussion