LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Action

2026-07-09

AI Summary

Existing vision–language–action (VLA) models exhibit limited performance in complex dynamic environments due to their uniform treatment of visual inputs and reliance on handcrafted heuristics, which hinder their ability to focus on task-critical evidence and model implicit environmental dynamics. To address this, this work proposes the LEEVLA architecture, which introduces a task-aware “where-to-attend–how-to-evolve” training framework. It incorporates Drift-Guided Dynamic Prioritization (DGDP) and Structured Feature Flow Generation (SFFG) to enable dynamic identification of task-relevant regions and structured evolution of latent representations. The approach integrates dynamic spatial prioritization, semantic drift guidance, prototype-to-boundary prediction, and mutual neighborhood contrastive loss. Evaluated across multiple VLA benchmarks, LEEVLA significantly outperforms current methods, demonstrating that explicit task-driven attention and structured latent reasoning are crucial for scalable VLA systems.

Abstract

Vision-language-action (VLA) models aim to map multimodal inputs to robot actions. However, most existing approaches struggle to cover complex dynamic scenarios due to treating all visual tokens uniformly and reasoning with human-selected factors, which lack mechanisms to emphasize task-critical evidence and ignore underlying factors. To address this issue, we propose LEEVLA, a VLA architecture for seeing what matters in Latent Environment Evolution that explicitly guides the model toward informative regions while preserving the structured evolution of latent world representations. To identify salient and instruction-relevant regions, we introduce drift-guided dynamic prioritization (DGDP), which combines dynamic position prioritization (DPP) with semantic drift guidance (SDG) to guide the VLA agent where to attend during training. On top of this, we introduce structured feature flow generation (SFFG), which models how these prioritized features should evolve in latent space via prototype-to-periphery (P2P) prediction, and a mutual-neighborhood contrastive (MC) loss to maintain topological consistency among neighborhoods. Together, DGDP and SFFG form a task-aware "where-how" training framework. Extensive experiments on VLA benchmarks show that LEEVLA consistently outperforms prior methods, confirming that explicit task-evidence guidance and structured latent reasoning are both crucial for scalable VLA. Our code is available at https://github.com/LyuQi127/LEEVLA.

Problem

• vision-language-action
• latent environment evolution
• task-critical evidence
• dynamic scenarios
• visual attention

Innovation

• drift-guided dynamic prioritization
• structured feature flow generation
• latent environment evolution
• vision-language-action
• mutual-neighborhood contrastive loss