We design "D-Twins" (Digital Twins), an LLM-based affective AI agent that embodies each user's emotional reactions and personality traits, presenting a real-time, authentic reflection of the user. D-Twins addresses the current lack of personalized boredom interventions in automated environments by utilizing real-time physiological data to provide interventions aligned with users' emotional responses. Initially, we collected users' natural language expressions to capture their unique characteristics. These patterns were used to create LLM-based AI agents that highly resemble the users. Then, we developed a boredom classification model by collecting electroencephalogram (EEG) data in an automated environment and integrated it into D-Twins. This integration enables D-Twins to rapidly recognize boredom and initiate personalized interventions, which users perceive as highly empathetic, turning boring environments into engaging experiences. Our study highlights that AI agents with user-similar emotional resonance offer a novel, real-time personalized intervention solution in boredom situations.
https://dl.acm.org/doi/10.1145/3706598.3714163
The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)