FlowMo-WM: A World Model with Object Momentum and Hidden Ambient Drift

2026-06-11

AI Summary

This work addresses the challenge of accurate long-term prediction in visual world models when object inertia and implicit environmental drift—such as water currents or wind—coexist. The authors propose FlowMo-WM, an end-to-end trainable visual world model that explicitly disentangles short-term object motion dynamics from slowly varying environmental drift contexts, thereby separating intrinsic dynamics from external force effects. A key innovation is the introduction of a zero-context residual transition mechanism, which enables unsupervised learning of disentangled representations without optical flow supervision. The effectiveness of this approach is validated through linear probing and context ablation studies. Evaluated on a simulated aquatic environment featuring diverse implicit flow fields and stochastic dynamics, FlowMo-WM significantly outperforms existing models in long-horizon rollout prediction accuracy.

Abstract

World models in robot learning predict future states from visual observations and actions, enabling agents to reason about the consequences of their controls. However, many action-conditioned models are evaluated in settings where motion is dominated by immediate control, whereas aquatic surface vehicles and other real-world objects continue moving under inertia and are displaced by hidden ambient drift, such as water currents or wind. We propose FlowMo-WM, an end-to-end trainable visual world model that infers object-centric motion state and a predictive long-history context associated with hidden drift from image-action histories without direct supervision of flow fields. FlowMo-WM factorizes image-action history into a short-history latent state, trained to summarize object-centric motion, and a longer-history context, trained to summarize slowly varying exogenous influences. A zero-context residual transition separates action-conditioned base dynamics from context-dependent drift effects during latent rollout. In simulated aquatic surface-vehicle environments with diverse hidden flows, disturbances, and randomized vehicle dynamics, FlowMo-WM improves long-horizon rollout accuracy over representative action-conditioned latent world models. Prediction-time context ablations, in which the inferred context is zeroed or shuffled during rollout, show that the ambient context is important for stable prediction under hidden drift, while frozen linear probes characterize information encoded in the learned factors.

Problem

• world models
• hidden ambient drift
• object momentum
• action-conditioned prediction
• visual observations

Innovation

• world model
• object-centric motion
• hidden ambient drift
• latent factorization
• long-horizon prediction