EventDrive: Event Cameras for Vision-Language Driving Intelligence

2026-06-16

AI Summary

This work addresses the challenge that existing vision-language models struggle to effectively integrate event camera data into the perception-to-decision loop for autonomous driving, particularly due to insufficient exploitation of temporal information in event streams under high-speed, motion-blur, or overexposure conditions. To bridge this gap, the study introduces event cameras into end-to-end driving intelligence for the first time, proposing a multi-temporal event pyramid and a time–field-of-view mixture-of-experts (MoE) architecture. This framework enables adaptive, multi-scale temporal modeling and fusion of asynchronous events with RGB frames, establishing a unified benchmark spanning perception, scene understanding, prediction, and planning. Experiments demonstrate significant improvements in temporal precision, motion awareness, and environmental robustness, confirming the critical role of event streams in enhancing high-level reasoning and decision-making in complex dynamic scenarios.

Abstract

Event cameras sense the world through asynchronous brightness changes with microsecond latency and high dynamic range, offering motion fidelity far beyond frame-based sensors and capturing temporal structure that conventional exposures often miss. These properties make events a powerful complement to RGB in autonomous driving, especially under blur, glare, and rapid motion, where frame-based perception can become unreliable. However, existing event-aware vision-language models remain limited to generic perception and do not reveal how event sensing contributes to reasoning and decision-making across the full driving loop. We present EventDrive, a large-scale benchmark and model suite that unifies event streams, RGB frames, and language supervision across four core dimensions: Perception, Understanding, Prediction, and Planning, covering captions, structured QA, grounding, motion-state recognition, trajectory forecasting, and planning tasks. Building on this foundation, EventDrive-VLM introduces a multi-horizon event pyramid and a temporal-horizon mixture-of-experts module to adaptively encode and fuse asynchronous and frame-based information for downstream reasoning. Comprehensive evaluation across diverse tasks shows that event streams provide substantial gains in temporal precision, motion awareness, and robustness, bringing event sensing into the center of driving intelligence.

Problem

• event cameras
• vision-language models
• autonomous driving
• driving intelligence
• temporal reasoning

Innovation

• event cameras
• vision-language models
• temporal-horizon mixture-of-experts
• multi-horizon event pyramid
• driving intelligence