SignSpeak: Open-Source Time Series Classification for ASL Translation

Aditya Makkar, Divya Makkar, Aarav Patel, Liam Hebert
6/27/2024
cs.CLcs.AIcs.LG

No official code found. Best implementation: adityamakkar000/SignSpeak (11 stars, MIT, last commit 2025-08-02). 1 more below.

Abstract

The lack of fluency in sign language remains a barrier to seamless communication for hearing and speech-impaired communities. In this work, we propose a low-cost, real-time ASL-to-speech translation glove and an exhaustive training dataset of sign language patterns. We then benchmarked this dataset with supervised learning models, such as LSTMs, GRUs and Transformers, where our best model achieved 92% accuracy. The SignSpeak dataset has 7200 samples encompassing 36 classes (A-Z, 1-10) and aims to capture realistic signing patterns by using five low-cost flex sensors to measure finger positions at each time step at 36 Hz. Our open-source dataset, models and glove designs, provide an accurate and efficient ASL translator while maintaining cost-effectiveness, establishing a framework for future work to build on.

View on arXivView PDF

Code Implementations(2)

adityamakkar000/SignSpeakCommunity100%
MIT

This repository contains the code and dataset for the paper "SignSpeak: Open-Source Time Series Classification for ASL Translation."

111C++, ShellDec 22, 20231 years agoMIT
Aarav-95/SignSpeakCommunity63%
MIT

This repository contains the code and dataset for the paper "SignSpeak: Open-Source Time Series Classification for ASL Translation."

10C++, ShellMar 28, 20251 years agoMIT

Similar Papers

A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Dataset

87% match2023

Using Sign Language Production as Data Augmentation to enhance Sign Language Translation

86% match2025

Sign Language Translation from Instructional Videos

85% match1 repo512023

From Rule-Based Models to Deep Learning Transformers Architectures for Natural Language Processing and Sign Language Translation Systems: Survey, Taxonomy and Performance Evaluation

85% match2024

FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluation

85% match2024

A Data-Driven Representation for Sign Language Production

84% match1 repo52024

Cite this paper

@article{makkar2024signspeak,
  title  = {SignSpeak: Open-Source Time Series Classification for ASL Translation},
  author = {Aditya Makkar and Divya Makkar and Aarav Patel and Liam Hebert},
  year   = {2024},
  eprint = {2407.12020},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.12020v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.12020.svg)](https://codeofpaper.com/paper/2407.12020)

Discussion