Satori 悟り: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling

要旨

Augmented Reality (AR) assistance is increasingly used for supporting users with physical tasks like assembly and cooking. However, most systems rely on reactive responses triggered by user input, overlooking rich contextual and user-specific information. To address this, we present Satori, a novel AR system that proactively guides users by modeling both -- their mental states and environmental contexts. Satori integrates the Belief-Desire-Intention (BDI) framework with the state-of-the-art multi-modal large language model (LLM) to deliver contextually appropriate guidance. Our system is designed based on two formative studies involving twelve experts. We evaluated the system with a sixteen within-subject study and found that Satori matches the performance of designer-created Wizard-of-Oz (WoZ) systems, without manual configurations or heuristics, thereby improving generalizability, reusability, and expanding the potential of AR assistance. Code is available at https://github.com/VIDA-NYU/satori-assistance.

著者
Chenyi Li
New York University, New York, New York, United States
Guande Wu
New York University, New York CIty, New York, United States
Gromit Yeuk-Yin Chan
Adobe Research, San Jose, California, United States
Dishita G. Turakhia
New York University, New York City, New York, United States
Sonia Castelo Quispe
New York University, New York, New York, United States
Dong Li
Tandon School of Engineering, Brooklyn, New York, United States
Leslie Welch
Brown University, Providence, Rhode Island, United States
Claudio Silva
New York University, New York City, New York, United States
Jing Qian
Tandon School of Engineering, Brooklyn, New York, United States
DOI

10.1145/3706598.3714188

論文URL

https://dl.acm.org/doi/10.1145/3706598.3714188

動画

会議: CHI 2025

The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)

セッション: XR Experience

Annex Hall F203
7 件の発表
2025-04-29 20:10:00
2025-04-29 21:40:00
日本語まとめ
読み込み中…