Si Chen
University of Illinois Urbana-Champaign, Champaign, Illinois, United States
Previous research underscored the potential of danmaku: a text-based commenting feature on videos for engaging hearing audiences. However, many Deaf and hard-of-hearing (DHH) users prioritize American Sign Language (ASL) over English. To improve inclusivity, we introduce Signmaku, a commenting mechanism that uses ASL as a sign language version of danmaku. Through a need-finding study (N=12) and a within-subject experiment (N=20), we evaluated three design styles: real human faces, cartoon-like, and robotic depictions. We found that cartoon signmaku not only provided entertainment but also prompted participants to create and share ASL comments with fewer privacy concerns compared to the other designs. Conversely, the robotic design's limited accuracy in conveying hand movements and facial expressions increased cognitive demands. Realist signmaku elicited the lowest cognitive load and was the easiest to understand among all three types. Our findings offer unique design implications for leveraging generative AI to create signmaku comments, enhancing co-learning experiences for DHH users.
doi.org/10.1145/3613904.3642287
The ACM CHI Conference on Human Factors in Computing Systems (https://chi2024.acm.org/)