Worlds in One Demo: A Synthetic Data Engine for Learning Open-World Mobile Manipulation

2026-07-14

AI Summary

This work addresses the challenges of open-world mobile manipulation policy learning, which is hindered by the high cost of real-world data collection and difficulties in achieving spatial generalization, long-horizon robustness, and cross-scenario transfer. The authors propose WANDA, a synthetic data engine that, given only a single human demonstration, reconstructs the environment via Gaussian splatting, re-simulates contact-rich interaction trajectories, performs whole-body motion planning, applies Corrective State Expansion, and generates diverse 3D scenes across environments to produce realistic and varied training data. This approach substantially improves long-horizon task success rates and enhances both spatial and cross-environment generalization in simulation and the real world, while enabling zero-shot transfer to mobile manipulators with different morphologies.

Abstract

Learning open-world mobile manipulation policies requires vast data to achieve spatial generalization, long-horizon robustness, and scene generalization. Current prevailing data collection paradigms, teleoperation and UMI, demand prohibitive human effort and cost at scale. To scale beyond the limits of manual data collection, we seek to maximize the value of each human demonstration by scalable data generation. To this end, we introduce WANDA: learning open-World mobile mANipulation from one demonstration via a synthetic DAta engine. WANDA first reconstructs background Gaussian splats and robot-object interaction trajectories from source RGBD observations, as a world substrate for later planning and rendering. It then rearranges contact-rich robot-object interaction segments into extensive spatial configurations, utilizing whole-body motion planning to chain them into new trajectories. To enhance long-horizon robustness, it applies Corrective State Expansion to increase the robot and object state diversity at different stages of mobile manipulation. To unlock cross-environment generalization, trajectories are synthesized on diverse generated 3D worlds from everyday photos. Furthermore, we synthesize photo-realistic observations by compositing rendered robot and object meshes with Gaussian splatting backgrounds. We evaluate our approach on extensive simulation and real-world tasks in various scenes. Experiments show that policies trained with WANDA achieve long-horizon robustness, broad spatial generalization and cross-environment generalization from one real demonstration. Moreover, WANDA naturally supports cross-embodiment data generation, validated by zero-shot deployment on another mobile manipulator with a distinct morphology.

Problem

• open-world mobile manipulation
• synthetic data generation
• spatial generalization
• long-horizon robustness
• cross-environment generalization

Innovation

• synthetic data generation
• Gaussian splatting
• mobile manipulation
• one-shot demonstration
• cross-environment generalization