UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving

2026-06-23

AI Summary

Existing approaches to scene understanding in autonomous driving struggle to simultaneously achieve robust temporal reasoning and high spatial precision, leading to inadequate dynamic risk perception and poor localization of small-scale or occluded hazardous objects. This work proposes UniDrive, a novel framework that explicitly integrates multi-frame temporal semantics with single-frame high-resolution perception. By employing gated cross-attention mechanisms, UniDrive effectively aligns dynamic contextual cues with fine-grained spatial details to jointly generate natural language risk descriptions and precise bounding boxes. Evaluated on the DRAMA-Reasoning benchmark, UniDrive significantly outperforms both image- and video-based baselines, achieving state-of-the-art performance in small object localization, zero-shot transfer to NuScenes and BDD100K, and human-rated interpretability and trustworthiness.

Abstract

Recent multimodal large language models (MLLMs) have shown strong potential for autonomous driving scene understanding, yet existing methods still face a fundamental trade-off between temporal reasoning and spatial precision. Models that rely on single-frame or low-resolution inputs often miss small, distant, or partially occluded hazards, while language-centric driving models frequently provide limited grounded evidence for their explanations. To address this gap, we propose UniDrive, a unified visual-language and grounding framework for interpretable risk understanding in autonomous driving. UniDrive combines a temporal reasoning branch that models scene dynamics from multi-frame visual input with a high-resolution perception branch that preserves fine-grained spatial details from the latest frame. The two branches are integrated through a gated cross-attention fusion module, enabling dynamic context to be aligned with precise spatial evidence. Based on the fused representation, UniDrive jointly generates natural-language risk descriptions and grounded bounding-box outputs for risk objects. Experiments on the DRAMA-Reasoning benchmark show that UniDrive outperforms representative image-based and video-based baselines in both captioning and risk-object grounding. In particular, UniDrive achieves the best overall performance on the validation split and demonstrates clear advantages in small-object localization, zero-shot generalization to NuScenes and BDD100K, and human-rated interpretability and trustworthiness. These results suggest that explicitly combining temporal semantics and high-resolution perception provides a stronger foundation for interpretable and safety-oriented autonomous driving systems. The code is available at https://github.com/pixeli99/unidrive-dev.

Problem

• autonomous driving
• risk understanding
• temporal reasoning
• spatial precision
• interpretable AI

Innovation

• temporal reasoning
• high-resolution perception
• grounded vision-language modeling
• gated cross-attention fusion
• interpretable autonomous driving