Foresight Without Seeing: Latent Futures for World Action Models

2026-08-11

AI Summary

Existing world models for robotic action struggle to simultaneously achieve efficient action prediction and explicit dynamics modeling during inference, often hindered by high computational costs or the absence of future state representations. This work proposes ForeWAM, an implicit future-state-driven direct-policy world model that provides dynamic context for action decisions without generating future videos. Its key innovations include a Future-KV mechanism that reuses key-value states from both visual inputs and future slots, and a dynamics register supervised by a frozen implicit action teacher to implicitly capture interaction-induced dynamics. Integrating Video DiT pre-filling with cross-layer state reuse, ForeWAM achieves 96.7% (standard) and 96.9% (accelerated) success rates on the LIBERO benchmark and 61.6% on LIBERO-Plus, all without requiring embodied pretraining.

Abstract

World Action Models (WAMs) couple future visual prediction with robot action generation, enabling policies to model how the physical world evolves during interaction. Existing WAMs differ in how predictive dynamics are exposed to the action pathway. Explicit-future WAMs provide direct access to predicted scene evolution, but incur substantial inference costs from iterative video denoising. In contrast, direct-policy WAMs efficiently predict actions from the current observation but lack an explicit inference-time interface for exposing predictive dynamics to the Action DiT. To bridge this gap, we propose ForeWAM, a dynamics-conditioned direct-policy WAM that provides predictive context for action generation without decoding future videos. At its core, Future-KV performs a single Video DiT prefill over the current visual latent and stochastic future slots, and reuses the resulting layer-wise key-value states throughout action denoising. We further introduce dynamics registers supervised by a frozen latent action teacher, encouraging the implicit future states to capture interaction-induced transitions such as object motion, contact changes, and task progress. Ground-truth future observations and the teacher are used only during training; deployment requires neither and performs no future video generation. Without embodied robot data pretraining, the standard and accelerated variants of ForeWAM achieve average success rates of 96.7% and 96.9% on LIBERO, respectively. The standard variant further achieves 61.6% success on LIBERO-Plus. These results demonstrate that direct-policy WAMs can retain efficient action prediction while exposing predictive dynamics to the action pathway without explicitly generating future observations.

Problem

• World Action Models
• predictive dynamics
• action generation
• future video prediction
• inference efficiency

Innovation

• World Action Models
• direct-policy WAM
• Future-KV
• dynamics registers
• latent future prediction