WithYou: Automated Adaptive Speech Tutoring With Context-Dependent Speech Recognition

要旨

Learning to speak in foreign languages is hard. Speech shadowing has been rising as a proven way to practice speaking, which asks a learner to listen and repeat a native speech template as simultaneously as possible. However, shadowing can be hard to do because learners can frequently fail to follow the speech and unintentionally interrupt a practice session. Worse, as a technical way to evaluate shadowing performance in real-time has not been established, no automated solutions are available to help. In this paper, we propose a technical framework with context-dependent speech recognition to evaluate shadowing in real-time. We propose a shadowing tutor system called WithYou, which can automatically adjust the playback and the difficulty of a speech template when learners fail, so shadowing becomes smooth and tailored. Results from a user study show that WithYou provides greater speech improvements (14%) than the conventional method (2.7%) with a lower cognitive load.

受賞
Honorable Mention
キーワード
Computer Assisted Language Learning (CALL)
Speaking
Shadowing
Speech Recognition
Intelligent Tutoring System, Language Learning
著者
Xinlei Zhang
University of Tokyo, Tokyo, Japan
Takashi Miyaki
University of Tokyo, Tokyo, Japan
Jun Rekimoto
University of Tokyo & Sony Computer Science Laboratories, Tokyo, Japan
DOI

10.1145/3313831.3376322

動画

会議: CHI 2020

The ACM CHI Conference on Human Factors in Computing Systems

セッション: Speech & language

Paper session
Paper
312 NI'IHAU
2020-04-29 20:00:00
2020-04-29 21:15:00
日本語まとめ
読み込み中…