AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systems

2025-04-08

AI Summary

To address insufficient interpretability and weak critical-region perception in autonomous driving decision-making, this paper proposes a cognitively aligned, end-to-end interpretable reinforcement learning (RL) framework. We introduce human gaze data—collected from 20 participants and comprising 1.2 million annotated frames—as an attention prior, and integrate it into the Proximal Policy Optimization (PPO) RL pipeline via a pre-trained attention-guidance mechanism, achieving cognitive alignment between agent decisions and human visual behavior across diverse driving scenarios. The method significantly enhances model robustness and interpretability: task completion rates improve by 23.6% across six complex driving scenarios, and critical-region localization accuracy increases by 31.4%. Our core contribution is the first large-scale, human-attention-guided interpretable RL paradigm for autonomous driving, establishing a novel pathway toward trustworthy, human-aligned decision-making in autonomous systems.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Reinforcement Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Improving decision-making capabilities in Autonomous Intelligent Vehicles (AIVs) has been a heated topic in recent years. Despite advancements, training machines to capture regions of interest for comprehensive scene understanding, like human perception and reasoning, remains a significant challenge. This study introduces a novel framework, Human Attention-based Explainable Guidance for Intelligent Vehicle Systems (AEGIS). AEGIS utilizes human attention, converted from eye-tracking, to guide reinforcement learning (RL) models to identify critical regions of interest for decision-making. AEGIS uses a pre-trained human attention model to guide RL models to identify critical regions of interest for decision-making. By collecting 1.2 million frames from 20 participants across six scenarios, AEGIS pre-trains a model to predict human attention patterns.

Problem

• Improving decision-making in Autonomous Intelligent Vehicles
• Training machines to mimic human attention for scene understanding
• Guiding RL models using human attention patterns for critical regions

Innovation

• Human attention guides reinforcement learning models
• Eye-tracking data converts to attention patterns
• Pre-trained model predicts human attention regions