ViSTAR: Virtual Skill Training with Augmented Reality with 3D Avatars and LLM coaching agent

要旨

We present ViSTAR, a Virtual Skill Training system in AR that supports self-guided basketball skill practice, with feedback on balance, posture, and timing. From a formative study with basketball players and coaches, the system addresses three challenges: understanding skills, identifying errors, and correcting mistakes. ViSTAR follows the Behavioral Skills Training (BST) framework—instruction, modeling, rehearsal, and feedback. It provides feedback through visual overlays, rhythm and timing cues, and an AI-powered coaching agent using 3D motion reconstruction. We generate verbal feedback by analyzing spatio-temporal joint data and mapping features to natural-language coaching cues via a Large Language Model (LLM). A key novelty is this feedback generation: motion features become concise coaching insights. In two studies (N=16), participants generally preferred our AI-generated feedback to coach feedback and reported that ViSTAR helped them notice posture and balance issues and refine movements beyond self-observation.

著者
Chunggi Lee
Harvard University, Cambridge, Massachusetts, United States
Hayato Saiki
University of Tsukuba, Tsukuba, Japan
Tica Lin
Dolby Laboratories, Atlanta, Georgia, United States
EIJI IKEDA
University of Tsukuba, Tsukuba, Japan
Kenji Suzuki
University of Tsukuba, Tsukuba, Japan
Chen Zhu-Tian
University of Minnesota-Twin Cities, Minneapolis, Minnesota, United States
Hanspeter Pfister
Harvard University, Cambridge, Massachusetts, United States

会議: CHI 2026

ACM CHI Conference on Human Factors in Computing Systems

セッション: Expression and Affective Wellbeing

P1 - Room 124
7 件の発表
2026-04-17 18:00:00
2026-04-17 19:30:00