A Skeleton-aware Graph Convolutional Network for Human-Object Interaction Detection

Manli Zhu, Edmond S. L. Ho, Hubert P. H. Shum
7/11/2022
cs.CVcs.AI

No official code found. Best implementation: zhumanli/SGCN4HOI (5 stars, MIT, last commit 2022-07-27).

Abstract

Detecting human-object interactions is essential for comprehensive understanding of visual scenes. In particular, spatial connections between humans and objects are important cues for reasoning interactions. To this end, we propose a skeleton-aware graph convolutional network for human-object interaction detection, named SGCN4HOI. Our network exploits the spatial connections between human keypoints and object keypoints to capture their fine-grained structural interactions via graph convolutions. It fuses such geometric features with visual features and spatial configuration features obtained from human-object pairs. Furthermore, to better preserve the object structural information and facilitate human-object interaction detection, we propose a novel skeleton-based object keypoints representation. The performance of SGCN4HOI is evaluated in the public benchmark V-COCO dataset. Experimental results show that the proposed approach outperforms the state-of-the-art pose-based models and achieves competitive performance against other models.

View on arXivView PDF

Code Implementations(1)

zhumanli/SGCN4HOICommunity100%
MIT

SGCN4HOI: A Skeleton-aware Graph Convolutional Network for Human-Object Interaction Detection

52PythonJul 7, 20224 years agoMIT

Similar Papers

Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection

89% match2023

TMHOI: Translational Model for Human-Object Interaction Detection

87% match2023

HOKEM: Human and Object Keypoint-based Extension Module for Human-Object Interaction Detection

87% match2023

IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognition

87% match2022

Geometric Features Enhanced Human-Object Interaction Detection

86% match1 repo72024

Improvement of Human-Object Interaction Action Recognition Using Scene Information and Multi-Task Learning Approach

86% match2025

Cite this paper

@article{zhu2022skeletonaware,
  title  = {A Skeleton-aware Graph Convolutional Network for Human-Object Interaction Detection},
  author = {Manli Zhu and Edmond S. L. Ho and Hubert P. H. Shum},
  year   = {2022},
  eprint = {2207.05733},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2207.05733v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2207.05733.svg)](https://codeofpaper.com/paper/2207.05733)

Discussion