Event-VLA: Action-Conditioned Event Fusion for Robust Vision-Language-Action Model

2026-06-28

AI Summary

This work addresses the significant performance degradation of existing vision-language-action (VLA) models in low-light or dynamically lit environments due to deteriorated RGB observations. To mitigate this limitation, the authors propose an event-augmented VLA framework that incorporates event streams—known for their illumination robustness and motion sensitivity—as a complementary modality. Central to the approach is an action query routing mechanism that employs learnable action queries and gated cross-attention to selectively fuse event information. This design preserves pretrained RGB-language semantic priors while enhancing the robustness of action representations. Experimental results demonstrate that the method maintains high performance under normal lighting conditions and substantially improves task success rates in low-light and near-dark scenarios, with validation provided through both simulated and real-world robotic experiments.

Abstract

Vision-Language-Action (VLA) models have become an important paradigm of embodied AI. However, existing VLA models typically assume well-lit and stable indoor settings, while real-world embodied manipulation may involve degraded RGB observations caused by illumination shifts, posing critical challenges for robust robotic manipulation. To address this gap, we propose \textbf{Event-VLA}, an event-enhanced VLA framework for generalizable manipulation across varying illumination conditions. We formulate VLA-based manipulation under degraded visibility as a practical robustness problem for RGB-centric policies, and introduce event streams as an illumination-robust, motion-sensitive complementary observation to improve robustness across visibility levels. Specifically, unlike conventional multimodal fusion that directly merges event features into the global semantic token space, Event-VLA injects event information through an action-query routing pathway. It uses learnable action queries to extract task-relevant semantics from the VLA reasoning process, and selectively aggregates event tokens via gated cross-attention to construct event-aware action representations. This design preserves the pretrained RGB-language semantic priors while effectively leveraging event information for robust action prediction. Experiments in simulation and real-world deployment show that Event-VLA maintains strong manipulation performance under normal lighting and improves success rates under low-light degradation and near-dark real-world settings.

Problem

• Vision-Language-Action
• robustness
• illumination degradation
• event camera
• embodied AI

Innovation

• Event Camera
• Vision-Language-Action
• Action-Conditioned Fusion
• Illumination Robustness
• Gated Cross-Attention