DSWAM: A Dual-System World Action Foundation Model for Fine-Grained Robot Manipulation

2026-07-06

AI Summary

This work addresses the limitations of existing World Action Models (WAMs) in decomposing coarse-grained instructions for complex, multi-step household tasks and the absence of a fair benchmark for comparing WAMs with Vision-Language-Action (VLA) policies on real robots. The authors propose a dual-system architecture activated on demand: System 1, grounded in video-based world modeling, generates fine-grained actions by default, while System 2—a vision-language planner—is invoked only when high-level instruction decomposition is required. This approach uniquely integrates WAM’s dense perceptual execution capabilities with VLA’s language-level planning strengths within a unified experimental framework enabling equitable comparison. Leveraging action-video co-training, TensorRT acceleration, and real-time action chunking, the system achieves efficient, real-time control. On the DeMaVLA deformable object manipulation benchmark, the method matches or exceeds VLA performance in fine manipulation under identical platforms, data, and evaluation protocols.

Abstract

World Action Models (WAMs) provide a promising alternative to Vision-Language-Action (VLA) policies by using video-based world modeling as dense supervision for robot action learning. Existing WAMs excel at physically grounded execution, but typically lack the explicit language-level planning interface in VLM-based VLAs for decomposing coarse instructions. Such decomposition becomes important when household tasks involve complex multi-step goals, where coarse user commands need to be converted into sequences of fine-grained executable subtasks. Meanwhile, the field still lacks a fair real-robot comparison between VLA and WAM execution capabilities, since existing systems often differ in data, robot embodiments, and task protocols. To address both the decomposition gap and the need for a controlled WAM-VLA comparison, we introduce DSWAM, a Dual-System World Action Foundation Model for fine-grained robot manipulation. DSWAM keeps a System 1 WAM executor as the default control path and optionally activates a System 2 vision-language subtask planner only when task decomposition is useful. The planner predicts executable subtasks from short-term visual history and a global task prompt, while the WAM executor performs world-aware action generation for each instruction or subtask. The executor is trained with action prediction and video co-training, but inference directly predicts action chunks without explicit future video generation. To make this execution path practical on real robots, we further integrate TensorRT acceleration, asynchronous execution, and real-time chunking (RTC) so that policy queries do not block robot control. To provide a fair real-robot comparison with VLA policies, we build and evaluate DSWAM under the DeMaVLA real-world deformable manipulation setting with matched robot platform, pretraining data, post-training data, and evaluation criteria.

Problem

• World Action Models
• Vision-Language-Action policies
• task decomposition
• fine-grained manipulation
• real-robot evaluation

Innovation

• World Action Model
• Dual-System Architecture
• Task Decomposition
• Real-Robot Evaluation
• Fine-Grained Manipulation