SpellRing: Recognizing Continuous Fingerspelling in American Sign Language using a Ring

要旨

Fingerspelling is a critical part of American Sign Language (ASL) recognition and has become an accessible optional text entry method for Deaf and Hard of Hearing (DHH) individuals. In this paper, we introduce SpellRing, a single smart ring worn on the thumb that recognizes words continuously fingerspelled in ASL. SpellRing uses active acoustic sensing (via a microphone and speaker) and an inertial measurement unit (IMU) to track handshape and movement, which are processed through a deep learning algorithm using Connectionist Temporal Classification (CTC) loss. We evaluated the system with 20 ASL signers (13 fluent and 7 learners), using the MacKenzie-Soukoref Phrase Set of 1,164 words and 100 phrases. Offline evaluation yielded top-1 and top-5 word recognition accuracies of 82.45% (±9.67%) and 92.42% (±5.70%), respectively. In real-time, the system achieved a word error rate (WER) of 0.099 (±0.039) on the phrases. Based on these results, we discuss key lessons and design implications for future minimally obtrusive ASL recognition wearables.

著者
Hyunchul Lim
Cornell, ITACA, New York, United States
Nam Anh Dang
Cornell University, Ithaca, New York, United States
Dylan Lee
Cornell University, Ithaca, New York, United States
Tianhong Catherine. Yu
Cornell University, Ithaca, New York, United States
Jane Lu
Cornell University, Ithaca, New York, United States
Franklin Mingzhe Li
Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
Yiqi Jin
Cornell University, Ithaca, New York, United States
Yan Ma
Stony Brook University, Stony Brook, New York, United States
Xiaojun Bi
Stony Brook University, Stony Brook, New York, United States
Francois Guimbretiere
Cornell University, Ithaca, New York, United States
Cheng Zhang
Cornell University, ITHACA, New York, United States
DOI

10.1145/3706598.3713721

論文URL

https://dl.acm.org/doi/10.1145/3706598.3713721

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: Assistive Technologies

G314+G315
7 件の発表
2025-05-01 01:20:00
2025-05-01 02:50:00
日本語まとめ
読み込み中…