S-squared-VLA: Decoupling Semantic and Spatial Streams in Vision-Language-Action Models for Autonomous Driving

2026-07-15

AI Summary

This work addresses the challenge in traditional vision-language-action (VLA) models where entangled semantic and spatial representations often lead to spatial representation collapse, hindering the generation of precise and safe control commands. To overcome this, the authors propose S²-VLA, a novel framework that explicitly decouples semantic and spatial streams within VLA for the first time. The semantic stream leverages a multi-scale vision-language model for intent reasoning, while the spatial stream bypasses linguistic bottlenecks to preserve raw visual geometric information. These two streams are integrated via a cascaded dual-stream attention mechanism. Auxiliary perceptual supervision is introduced to maintain geometric priors, and the entire system is fine-tuned end-to-end. Evaluated on the NAVSIM closed-loop benchmark, S²-VLA achieves a new state-of-the-art PDMS score of 87.1 under purely supervised fine-tuning and attains the lowest collision rate, with 98.4% of trajectories collision-free.

Abstract

Vision-Language Models (VLMs) have demonstrated remarkable potential for high-level reasoning in autonomous driving, yet they fundamentally struggle to generate precise, low-level control actions. This limitation is rooted in a semantic-physical gap caused by the inherent mismatch between discrete language tokens and continuous trajectory planning. While Vision-Language-Action (VLA) architectures attempt to bridge this gap by unifying perception and control into a single policy, this entanglement creates a new bottleneck. Standard VLAs experience a severe spatial representation collapse, which irreversibly degrades the fine-grained spatial and geometric priors essential for safe, boundary-aware navigation. To address this limitation, we propose the S-squared-VLA, which explicitly decouples the semantic and spatial streams in Vision-Language-Action models. The semantic stream leverages hierarchical bridging to extract multi-scale VLM features for robust intent reasoning. In parallel, an independent spatial stream bypasses the autoregressive language bottleneck, directly preserving uncompressed spatial features from the visual encoder. By integrating auxiliary perception supervision, this stream explicitly equips the model with rich spatial and geometric priors. Finally, a dual-stream planning adapter fuses high-level semantic intent with precise spatial constraints via cascaded attention mechanisms. Evaluations on the NAVSIM closed-loop benchmark show that S-squared-VLA achieves a Predictive Driver Model Score (PDMS) of 87.1, establishing a new state-of-the-art for VLA models under a purely supervised fine-tuning (SFT) setting. By mitigating the spatial representation collapse of traditional VLMs, our framework significantly outperforms baselines, achieving the highest No Collision (NC) rate of 98.4 among all evaluated methods.

Problem

• Vision-Language-Action
• spatial representation collapse
• semantic-physical gap
• autonomous driving
• trajectory planning

Innovation

• Vision-Language-Action
• spatial representation decoupling
• geometric priors
• dual-stream architecture
• autonomous driving