Decoupling Vision, Language, and Action for Efficient Multi-Task Robot Policies

2026-09-16

AI Summary

研究通过解耦视觉、语言和动作模块，提出Decoupled Embodiment Model (DEM)，在保持任务成功率的同时，大幅提高推理速度并降低能耗。

Abstract

Vision-Language-Action (VLA) models attach an action module to a Vision-Language Model (VLM) with billions of parameters and pay for that backbone at every control step. For a low-level manipulation policy, this cost may be unnecessary: the VLM supplies vision and language embeddings, and recent standalone vision encoders and encoder-only language models now match or exceed large VLMs on visual embedding and language understanding benchmarks. We study this question with a controlled experiment. Holding the demonstrations, the training budget, the tasks, and the measurement platform fixed, we vary the vision encoder, the language encoder, and the action head of a decoupled policy and compare against seven VLA baselines. The study yields the Decoupled Embodiment Model (DEM), which pairs a fine-tuned DINOv3 encoder and a frozen NeoBERT encoder with a MeanFlow head that generates each action chunk in a single forward pass. On 18 simulated manipulation tasks with held-out language paraphrases and randomized scenes, and on three real-robot tasks, DEM achieves observed success comparable to state-of-the-art VLM-backbone policies under our evaluation protocol, while running at eight to seventeen times their inference frequency and drawing six to fifteen times less energy per inference. Within this task scope, modern decoupled components offer a better success--latency--energy trade-off.

Problem

• Vision-Language-Action
• Efficiency
• Multi-Task Robot Policies
• Decoupling
• Inference Frequency

Innovation

• Decoupled Embodiment Model
• DINOv3
• NeoBERT
• MeanFlow head
• multi-task robot policies