M^2Silent: Enabling Multi-user Silent Speech Interactions via Multi-directional Speakers in Shared Spaces

要旨

We introduce M^2Silent, which enables multi-user silent speech interactions in shared spaces using multi-directional speakers. Ensuring privacy during interactions with voice-controlled systems presents significant challenges, particularly in environments with multiple individuals, such as libraries, offices, or vehicles. M^2Silent addresses this by allowing users to communicate silently, without producing audible speech, using acoustic sensing integrated into directional speakers. We leverage FMCW signals as audio carriers, simultaneously playing audio and sensing the user's silent speech. To handle the challenge of multiple users interacting simultaneously, we propose time-shifted FMCW signals and blind source separation algorithms, which help isolate and accurately recognize the speech features of each user. We also present a deep-learning model for real-time silent speech recognition. M^2Silent achieves Word Error Rate (WER) of 6.5% and Sequence Error Rate (SER) of 12.8% in multi-user silent speech recognition while maintaining high audio quality, offering a novel solution for privacy-preserving, multi-user silent interactions in shared spaces.

著者
Juntao Zhou
Shanghai Jiao Tong University, Shanghai, China
Dian Ding
Shanghai Jiao Tong University, Shanghai, China
Yijie Li
National University of Singapore, Singapore, Singapore
Yu Lu
Shanghai Jiao Tong University, Shanghai, China
Yida Wang
Shanghai Jiao Tong University, Shanghai, Shanghai, China
Yongzhao Zhang
University of Electronic Science and Technology of China, Chengdu, Sichuan, China
Yi-Chao Chen
Shanghai Jiao Tong University, Shanghai, China
Guangtao Xue
Shanghai Jiao Tong University, Shanghai, China
DOI

10.1145/3706598.3714174

論文URL

https://dl.acm.org/doi/10.1145/3706598.3714174

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: Multimodal Interaction

G302
7 件の発表
2025-04-30 18:00:00
2025-04-30 19:30:00
日本語まとめ
読み込み中…