Sign Language Translation from Instructional Videos

Laia Tarrés, Gerard I. Gállego, Amanda Duarte, Jordi Torres, Xavier Giró-i-Nieto
4/13/2023
cs.CLcs.CV

No official code found. Best implementation: imatge-upc/slt_how2sign_wicv2023 (51 stars, MIT, last commit 2023-07-23).

Abstract

The advances in automatic sign language translation (SLT) to spoken languages have been mostly benchmarked with datasets of limited size and restricted domains. Our work advances the state of the art by providing the first baseline results on How2Sign, a large and broad dataset. We train a Transformer over I3D video features, using the reduced BLEU as a reference metric for validation, instead of the widely used BLEU score. We report a result of 8.03 on the BLEU score, and publish the first open-source implementation of its kind to promote further advances.

View on arXivView PDF

Code Implementations(1)

imatge-upc/slt_how2sign_wicv2023Community100%
MIT

Sign Language Translation for Instructional Videos - CVPR WiCV 2023

5113C++, LuaApr 11, 20233 years agoMIT

Similar Papers

Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation

89% match1 repo462026

SyriSign: A Parallel Corpus for Arabic Text to Syrian Arabic Sign Language Translation

89% match2026

SignBank+: Preparing a Multilingual Sign Language Dataset for Machine Translation Using Large Language Models

89% match1 repo222023

A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars

88% match1 repo3932026

A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Dataset

88% match2023

Scaling Sign Language Translation

88% match2024

Cite this paper

@article{tarrs2023sign,
  title  = {Sign Language Translation from Instructional Videos},
  author = {Laia Tarrés and Gerard I. Gállego and Amanda Duarte and Jordi Torres and Xavier Giró-i-Nieto},
  year   = {2023},
  eprint = {2304.06371},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2304.06371v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.06371.svg)](https://codeofpaper.com/paper/2304.06371)

Discussion