Improving decision-making capabilities in Autonomous Intelligent Vehicles (AIVs) has been a heated topic in recent years. Despite advancements, training machine to capture regions of interest for comprehensive scene understanding, like human perception and reasoning, remains a significant challenge. This study introduces a novel framework, Human Attention-based Explainable Guidance for Intelligent Vehicle Systems (AEGIS). AEGIS utilizes human attention, converted from eye-tracking, to guide reinforcement learning (RL) models to identify critical regions of interest for decision-making. AEGIS uses a pre-trained human attention model to guide reinforcement learning (RL) models to identify critical regions of interest for decision-making. By collecting 1.2 million frames from 20 participants across six scenarios, AEGIS pre-trains a model to predict human attention patterns. The learned human attention guides the RL agent’s focus on task-relevant objects, prioritizes critical instances, enhances robustness in unseen environments, and leads to faster learning convergence. This approach enhances interpretability by making machine attention more comparable to human attention and thus enhancing the RL agent’s performance in diverse driving scenarios. The code is available in https://github.com/ALEX95GOGO/AEGIS.
https://dl.acm.org/doi/10.1145/3706598.3713779
The ACM CHI Conference on Human Factors in Computing Systems (https://chi2025.acm.org/)