SLVideo: A Sign Language Video Moment Retrieval Framework

Gonçalo Vinagre Martins, João Magalhães, Afonso Quinaz, Carla Viegas, Sofia Cavaco
7/22/2024
cs.CVcs.AI

No confident code implementation found for this paper.

Abstract

SLVideo is a video moment retrieval system for Sign Language videos that incorporates facial expressions, addressing this gap in existing technology. The system extracts embedding representations for the hand and face signs from video frames to capture the signs in their entirety, enabling users to search for a specific sign language video segment with text queries. A collection of eight hours of annotated Portuguese Sign Language videos is used as the dataset, and a CLIP model is used to generate the embeddings. The initial results are promising in a zero-shot setting. In addition, SLVideo incorporates a thesaurus that enables users to search for similar signs to those retrieved, using the video segment embeddings, and also supports the edition and creation of video sign language annotations. Project web page: https://novasearch.github.io/SLVideo/

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

SEDS: Semantically Enhanced Dual-Stream Encoder for Sign Language Retrieval

86% match1 repo72024

CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning

83% match1 repo3932023

SignIT: A Comprehensive Dataset and Multimodal Analysis for Italian Sign Language Recognition

83% match2025

QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries

82% match1 repo3542021

A Survey on Video Moment Localization

82% match2023

Mitigating Modality and Language-Style Gaps for Zero-Shot Video Moment Retrieval

81% match2026

Cite this paper

@article{martins2024slvideo,
  title  = {SLVideo: A Sign Language Video Moment Retrieval Framework},
  author = {Gonçalo Vinagre Martins and João Magalhães and Afonso Quinaz and Carla Viegas and Sofia Cavaco},
  year   = {2024},
  eprint = {2407.15668},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.15668v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.15668.svg)](https://codeofpaper.com/paper/2407.15668)

Discussion