Risk Map As Middleware: Towards Interpretable Cooperative End-to-end Autonomous Driving for Risk-Aware Planning

2025-08-11

AI Summary

End-to-end autonomous driving for single agents suffers from hazardous behaviors due to perceptual occlusions and limited field-of-view, compounded by the lack of interpretability in black-box decision-making. Method: This paper proposes a collaborative end-to-end framework grounded in a learnable risk map. It introduces risk maps as an interpretable intermediate representation between perception and planning, explicitly modeling spatiotemporal multi-vehicle interaction risks. A unified Transformer architecture with attention mechanisms extracts risk-aware features, while a learned model predictive control (MPC) module enables interpretable trajectory planning. Contribution/Results: Evaluated on the real-world V2XPnP-Seq dataset, the framework significantly improves risk perception accuracy, planning robustness, and safety. Crucially, it provides transparent, human-understandable behavioral decision rationales—addressing both performance and explainability gaps in end-to-end autonomous driving.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

End-to-end paradigm has emerged as a promising approach to autonomous driving. However, existing single-agent end-to-end pipelines are often constrained by occlusion and limited perception range, resulting in hazardous driving. Furthermore, their black-box nature prevents the interpretability of the driving behavior, leading to an untrustworthiness system. To address these limitations, we introduce Risk Map as Middleware (RiskMM) and propose an interpretable cooperative end-to-end driving framework. The risk map learns directly from the driving data and provides an interpretable spatiotemporal representation of the scenario from the upstream perception and the interactions between the ego vehicle and the surrounding environment for downstream planning. RiskMM first constructs a multi-agent spatiotemporal representation with unified Transformer-based architecture, then derives risk-aware representations by modeling interactions among surrounding environments with attention. These representations are subsequently fed into a learning-based Model Predictive Control (MPC) module. The MPC planner inherently accommodates physical constraints and different vehicle types and can provide interpretation by aligning learned parameters with explicit MPC elements. Evaluations conducted on the real-world V2XPnP-Seq dataset confirm that RiskMM achieves superior and robust performance in risk-aware trajectory planning, significantly enhancing the interpretability of the cooperative end-to-end driving framework. The codebase will be released to facilitate future research in this field.

Problem

• Enhance interpretability in autonomous driving systems
• Address occlusion and perception range limitations
• Improve risk-aware trajectory planning performance

Innovation

• Risk Map as Middleware for interpretable driving
• Transformer-based multi-agent spatiotemporal representation
• Learning-based MPC with physical constraints