Event-RGB Adaptive Tracking for Nighttime Highway Perception

2026-07-13

AI Summary

This work addresses the significant degradation in perception performance under nighttime highway conditions, where RGB cameras suffer from motion blur, underexposure, and low signal-to-noise ratio due to insufficient illumination and high-speed motion. To overcome these limitations, we propose the Joint Event-RGB Adaptive Tracking (JEAT) framework, which employs an adaptive extended Kalman filter to dynamically estimate modality-specific noise through Normalized Innovation Squared (NIS) statistics, enabling asynchronous and soft-weighted fusion of event streams and RGB frames. This approach transcends the rigid, hard-coded priority schemes typical of conventional multi-sensor tracking systems. Furthermore, we introduce SEHN, the first synchronized synthetic event-RGB dataset tailored for diverse highway environments, generated using the CARLA simulation platform. Experimental results demonstrate that JEAT substantially enhances the robustness and accuracy of vehicle tracking in extreme nighttime scenarios on the SEHN dataset.

Abstract

Intelligent Transportation Systems deployed on highways predominantly rely on conventional RGB cameras for traffic perception and vehicle tracking. However, highway environments present unique challenges: the absence of artificial lighting infrastructure, combined with high vehicle velocities, results in severely degraded perception performance under low-light conditions. Specifically, nighttime scenarios suffer from motion blur, insufficient exposure, and poor signal-to-noise ratios, which catastrophically impair the reliability of RGB-based sensing systems. To address these limitations, we propose a novel Joint Event-RGB Adaptive Tracking (JEAT) framework. Unlike existing multi-sensor trackers constrained by rigid, hard-coded prioritization, JEAT merges asynchronous event streams and RGB frames into a unified joint data association optimization. By employing an Adaptive Extended Kalman Filter to continuously estimate measurement noise via NIS statistics, the framework dynamically weights and fuses both modalities, optimally harnessing event streams during dark or high-speed motion while leveraging RGB frames under bright or static conditions. Furthermore, given the absence of publicly available datasets tailored for event-based highway perception with diverse environmental conditions, we present SEHN, a large-scale synthetic dataset generated using the CARLA simulator. Our dataset encompasses diverse environmental conditions (daytime, nighttime, nighttime with out artificial lighting) and varying traffic densities, providing synchronized RGB imagery and event streams to facilitate multi-modal fusion research. Our code and datasets will be available at https://github.com/haidongwang96/SEHN.

Problem

• nighttime perception
• highway tracking
• low-light conditions
• RGB camera limitations
• motion blur

Innovation

• Event-based vision
• Adaptive sensor fusion
• Multi-modal tracking
• Nighttime perception
• Synthetic dataset