Unknown Word Detection for English as a Second Language (ESL) Learners using Gaze and Pre-trained Language Models

要旨

English as a Second Language (ESL) learners often encounter unknown words that hinder their text comprehension. Automatically detecting these words as users read can enable computing systems to provide just-in-time definitions, synonyms, or contextual explanations, thereby helping users learn vocabulary in a natural and seamless manner. This paper presents EyeLingo, a transformer-based machine learning method that predicts the probability of unknown words based on text content and eye gaze trajectory in real time with high accuracy. A 20-participant user study revealed that our method can achieve an accuracy of 97.6%, and an F1-score of 71.1%. We implemented a real-time reading assistance prototype to show the effectiveness of EyeLingo. The user study shows improvement in willingness to use and usefulness compared to baseline methods.

著者
Jiexin Ding
Tsinghua University, Beijing, China
Bowen Zhao
Groundlight AI, Seattle, Washington, United States
Yuntao Wang
Tsinghua University, Beijing, China
Xinyun Liu
Rice University, Houston, Texas, United States
Rui Hao
University of Chinese Academy of Sciences, Beijing, China
Ishan Chatterjee
University of Washington, Seattle, Washington, United States
Yuanchun Shi
Tsinghua University, Beijing, China
DOI

10.1145/3706598.3714181

論文URL

https://dl.acm.org/doi/10.1145/3706598.3714181

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: Language Matters

G318+G319
6 件の発表
2025-05-01 18:00:00
2025-05-01 19:30:00
日本語まとめ
読み込み中…