Learning 4D Geometric Priors for Inference-Efficient World Action Models

2026-07-06

AI Summary

Existing world action models struggle to effectively capture task-relevant spatiotemporal geometric information when jointly modeling visual dynamics and executable actions, limiting manipulation accuracy. This work proposes MECo-WAM, a multi-expert collaborative training framework that enhances video-action representations through action-aware 4D geometric priors. Key innovations include an asymmetric expert visibility mechanism to prevent non-causal shortcuts, a decayed 4D readout masking attention module, and an action-aware temporal geometric distillation strategy that injects geometric knowledge during training while enabling zero-overhead deployment at inference. Experiments demonstrate significant performance gains on LIBERO (98.2%), RoboTwin 2.0 (92.6%), and complex real-world manipulation tasks, all without increasing inference computational cost.

Abstract

World Action Models (WAMs) have shown strong potential for robotic manipulation by jointly modeling visual future dynamics and executable action sequences. However, existing video-action co-training methods primarily optimize appearance-oriented video latents, which may insufficiently capture the temporally evolving geometry required for precise manipulation. We propose MECo-WAM, a Multi-Expert Co-Training World Action Model that injects action-relevant 4D geometric priors into video-action representations while preserving the original lightweight inference graph. During training, MECo-WAM combines video and action experts with a lightweight 4D expert supervised by relational targets from a frozen VGGT encoder. Asymmetric expert visibility prevents non-causal shortcuts from auxiliary geometry to action generation. To transfer geometric knowledge into the deployed video-action pathway, we introduce decayed 4D read-mask attention, which provides restricted current-frame geometric guidance early in training and progressively removes this dependency. We further propose action-aware temporal geometric distillation, which aligns within-frame geometric relations and their temporal evolution while emphasizing visual regions most relevant to robot actions. At deployment, all auxiliary 4D components are removed. Experiments on LIBERO (98.2%), RoboTwin 2.0 (92.6%), and challenging real-world manipulation tasks show that MECo-WAM improves manipulation performance without increasing inference cost.

Problem

• World Action Models
• 4D geometric priors
• robotic manipulation
• video-action co-training
• temporal geometry

Innovation

• 4D geometric priors
• multi-expert co-training
• inference-efficient
• temporal geometric distillation
• world action models