Towards Driver Behavior Understanding: Weakly-Supervised Risk Perception in Driving Scenes

2026-03-06

AI Summary

This study addresses the challenge of enhancing zero-collision capabilities in intelligent vehicles by modeling drivers’ perception of traffic risk under conditions of limited fine-grained annotations. To this end, we introduce RAID, the first large-scale, multidimensional dataset encompassing driver intention, pedestrian attention, and risk contexts. We further propose a weakly supervised framework for risk-object identification that localizes risk sources without per-frame annotations by leveraging the association between driver intention and responsive behavior. Integrating video-based behavioral analysis, intention modeling, and contextual risk assessment, our method outperforms state-of-the-art approaches by 20.6% on the RAID dataset and by 23.1% on the HDDS dataset, demonstrating the effectiveness of both the proposed dataset and methodology.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Video Understanding & Activity Analysis
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Achieving zero-collision mobility remains a key objective for intelligent vehicle systems, which requires understanding driver risk perception-a complex cognitive process shaped by voluntary response of the driver to external stimuli and the attentiveness of surrounding road users towards the ego-vehicle. To support progress in this area, we introduce RAID (Risk Assessment In Driving scenes)-a large-scale dataset specifically curated for research on driver risk perception and contextual risk assessment. RAID comprises 4,691 annotated video clips, covering diverse traffic scenarios with labels for driver's intended maneuver, road topology, risk situations (e.g., crossing pedestrians), driver responses, and pedestrian attentiveness. Leveraging RAID, we propose a weakly supervised risk object identification framework that models the relationship between driver's intended maneuver and responses to identify potential risk sources. Additionally, we analyze the role of pedestrian attention in estimating risk and demonstrate the value of the proposed dataset. Experimental evaluations demonstrate that our method achieves 20.6% and 23.1% performance gains over prior state-of-the-art approaches on the RAID and HDDS datasets, respectively.

Problem

• driver behavior
• risk perception
• weakly-supervised learning
• driving scenes
• pedestrian attentiveness

Innovation

• weakly-supervised learning
• driver risk perception
• risk object identification
• pedestrian attentiveness
• driving scene understanding