LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving

2026-03-02

AI Summary

This work addresses the limitations of existing vision-language-action (VLA) models, which rely on explicit textual reasoning and often suffer from semantic-perceptual misalignment and symbol-grounding conflicts. To overcome these issues, we propose LaST-VLA, a novel framework that, for the first time, models reasoning as an implicit spatiotemporal continuum governed by physical constraints. LaST-VLA integrates 3D geometric priors with dynamic predictions through a dual-feature alignment mechanism and jointly embeds geometric and dynamic priors in a latent space, thereby circumventing the fragmentation inherent in conventional chain-of-thought approaches. Combining 3D foundation models, world models, progressive supervised fine-tuning, and GRPO reinforcement learning, our method achieves state-of-the-art performance on NAVSIM v1 (91.3 PDMS) and v2 (87.1 EPDMS), and demonstrates exceptional spatiotemporal reasoning capabilities on the SURDS and NuDynamics benchmarks.

Technology Category

• Computer Vision: Language and Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

While Vision-Language-Action (VLA) models have revolutionized autonomous driving by unifying perception and planning, their reliance on explicit textual Chain-of-Thought (CoT) leads to semantic-perceptual decoupling and perceptual-symbolic conflicts. Recent shifts toward latent reasoning attempt to bypass these bottlenecks by thinking in continuous hidden space. However, without explicit intermediate constraints, standard latent CoT often operates as a physics-agnostic representation. To address this, we propose the Latent Spatio-Temporal VLA (LaST-VLA), a framework shifting the reasoning paradigm from discrete symbolic processing into a physically grounded Latent Spatio-Temporal CoT. By implementing a dual-feature alignment mechanism, we distill geometric constraints from 3D foundation models and dynamic foresight from world models directly into the latent space. Coupled with a progressive SFT training strategy that transitions from feature alignment to trajectory generation, and refined via Reinforcement Learning with Group Relative Policy Optimization (GRPO) to ensure safety and rule compliance. \method~setting a new record on NAVSIM v1 (91.3 PDMS) and NAVSIM v2 (87.1 EPDMS), while excelling in spatial-temporal reasoning on SURDS and NuDynamics benchmarks.

Problem

• Vision-Language-Action
• Chain-of-Thought
• Latent Reasoning
• Spatio-Temporal Reasoning
• Autonomous Driving

Innovation

• Latent Spatio-Temporal CoT
• Vision-Language-Action
• Dual-feature Alignment
• Group Relative Policy Optimization
• Physically Grounded Reasoning