LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies

2026-06-14

AI Summary

This work addresses the challenge that existing vision-language-action models struggle to explicitly predict scene dynamics induced by actions, while conventional world models rely on computationally expensive pixel-level video generation. To overcome this limitation, the authors propose LaWAM (Latent World Action Model), which for the first time integrates the latent space of pretrained vision foundation models with action-conditioned dynamics modeling. By leveraging lightweight latent dynamic prediction and compact latent visual subgoals to guide policy learning, LaWAM enables efficient and dynamics-aware robotic control. The method achieves state-of-the-art performance on the LIBERO (98.6%) and RoboTwin (91.22%) benchmarks as well as in real-world tasks, with a single action prediction requiring only 187 milliseconds—reducing latency by up to 24× compared to pixel-level world models.

Abstract

Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.

Problem

• Vision-Language-Action models
• World-Action Models
• predictive dynamics
• video generation
• robot control

Innovation

• Latent World Model
• Vision-Language-Action
• Dynamics-Aware Control
• Efficient Robot Policy
• Latent Visual Subgoals