UniFS: Unified Fast-to-Slow Hierarchical Architecture for Vision-Language-Action Models

2026-06-21

AI Summary

This work addresses the mismatch in update frequencies between fast and slow subsystems in existing vision-language-action models, which often leads to semantic drift or inefficiency, and the common practice of leveraging only a single-layer feature from vision-language models while neglecting richer intermediate semantic information. To overcome these limitations, the authors propose a unified fast-to-slow hierarchical architecture that assigns decreasing update frequencies across layers of the vision-language model. They introduce a latent vector inversion mechanism to align multi-scale features with action decoding and incorporate multi-level temporal supervision to enable coarse-to-fine cross-scale learning. The method achieves a 98.3% average success rate (+2.5%) on the LIBERO benchmark, reduces inference latency to 17.8 ms (a 2.1× speedup), and demonstrates effective real-world deployment on a Franka robotic platform.

Abstract

Mainstream Fast-Slow dual system vision-language-action models decouple a high-frequency action expert from a low-frequency vision-language model for efficiency, yet they face a fundamental frequency dilemma: large update gaps cause semantic drift from stale context, while small gaps erode the intended computational savings. Moreover, because the action expert receives only the VLM's final-layer representation at a single fixed frequency, rich intermediate features are discarded, limiting both information coupling and manipulation precision. Inspired by multi-timescale neural processing in the human brain, we introduce UniFS, a unified fast-to-slow architecture that resolves these challenges through three key designs. First, we stratify the VLM layers into groups with progressively decreasing update frequencies, enabling shallow layers to capture fast-changing dynamics while deeper layers cache stable semantic context. Second, a latent vector inversion mechanism re-routes the interaction order between multi-scale VLM features and the action expert, aligning fast-varying representations with fine-grained action decoding and slow-varying ones with coarse planning. Third, a multi-level supervision strategy enforces a coarse-to-fine learning hierarchy across temporal scales. Together, these designs enable richer cross-frequency information transfer within a single backbone, while the low-frequency pathways additionally preserve temporal context across steps. Experiments on LIBERO show that UniFS achieves state-of-the-art performance (98.3\% average success rate, a 2.5\% gain over VLA-Adapter baseline) while reducing average inference latency from 36.5~ms to 17.8~ms (2.1$\times$ speedup). Real-robot experiments on a Franka platform further validate its practical applicability. Code is opensourced at https://github.com/linsun449/UniFS.

Problem

• frequency dilemma
• semantic drift
• vision-language-action models
• information coupling
• temporal context

Innovation

• fast-to-slow architecture
• multi-timescale processing
• latent vector inversion
• hierarchical supervision
• vision-language-action models