Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising

2026-04-29

AI Summary

This work addresses the limitations of existing unified world models, which operate solely in 2D pixel space and struggle to balance action execution efficiency with high-quality 4D modeling. The authors propose X-WAM, the first unified 4D world model, built upon a pretrained video diffusion Transformer to enable high-fidelity 4D world synthesis and efficient action execution through multi-view RGB-D video prediction. Key innovations include a lightweight structural adaptation mechanism that repurposes the pretrained model for depth generation and an Asynchronous Noising Strategy (ANS) that differentially handles action decoding and video generation during inference. Experiments demonstrate that X-WAM achieves average success rates of 79.2% on RoboCasa and 90.7% on RoboTwin 2.0, with 4D reconstructions significantly outperforming prior methods across both visual and geometric evaluation metrics.

Abstract

We propose X-WAM, a Unified 4D World Model that unifies real-time robotic action execution and high-fidelity 4D world synthesis (video + 3D reconstruction) in a single framework, addressing the critical limitations of prior unified world models (e.g., UWM) that only model 2D pixel-space and fail to balance action efficiency and world modeling quality. To leverage the strong visual priors of pretrained video diffusion models, X-WAM imagines the future world by predicting multi-view RGB-D videos, and obtains spatial information efficiently through a lightweight structural adaptation: replicating the final few blocks of the pretrained Diffusion Transformer into a dedicated depth prediction branch for the reconstruction of future spatial information. Moreover, we propose Asynchronous Noise Sampling (ANS) to jointly optimize generation quality and action decoding efficiency. ANS applies a specialized asynchronous denoising schedule during inference, which rapidly decodes actions with fewer steps to enable efficient real-time execution, while dedicating the full sequence of steps to generate high-fidelity video. Rather than entirely decoupling the timesteps during training, ANS samples from their joint distribution to align with the inference distribution. Pretrained on over 5,800 hours of robotic data, X-WAM achieves 79.2% and 90.7% average success rate on RoboCasa and RoboTwin 2.0 benchmarks, while producing high-fidelity 4D reconstruction and generation surpassing existing methods in both visual and geometric metrics.

Problem

• unified world model
• 4D world modeling
• robotic action execution
• video diffusion
• real-time control

Innovation

• 4D World Modeling
• Asynchronous Denoising
• Video Diffusion Priors
• Robotic Action Execution
• Depth Prediction Branch