A Recipe for Efficient Sim-to-Real Transfer in Manipulation with Online Imitation-Pretrained World Models

2025-10-02

AI Summary

To address the limited coverage and sharp performance degradation of imitation learning caused by scarce expert demonstrations in real-world settings, this paper proposes an online-offline collaborative simulation-to-reality (sim-to-real) transfer framework. The method first pretrains a world-model-based policy in simulation, then performs lightweight online interaction in the real environment for policy calibration, followed by fine-tuning with a small amount of real-world expert data. This design effectively alleviates the data-coverage bottleneck inherent in purely offline approaches, significantly enhancing cross-domain generalization and robustness. Experimental results demonstrate that the proposed method achieves success rates 31.7% higher than state-of-the-art methods in sim-to-sim tasks and 23.3% higher in sim-to-real tasks, validating its efficiency and practicality.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

We are interested in solving the problem of imitation learning with a limited amount of real-world expert data. Existing offline imitation methods often struggle with poor data coverage and severe performance degradation. We propose a solution that leverages robot simulators to achieve online imitation learning. Our sim-to-real framework is based on world models and combines online imitation pretraining with offline finetuning. By leveraging online interactions, our approach alleviates the data coverage limitations of offline methods, leading to improved robustness and reduced performance degradation during finetuning. It also enhances generalization during domain transfer. Our empirical results demonstrate its effectiveness, improving success rates by at least 31.7% in sim-to-sim transfer and 23.3% in sim-to-real transfer over existing offline imitation learning baselines.

Problem

• Achieving efficient sim-to-real transfer in robot manipulation
• Overcoming data coverage limitations in offline imitation learning
• Improving robustness and generalization during domain transfer

Innovation

• Online imitation pretraining with world models
• Sim-to-real transfer via offline finetuning
• Leveraging simulator interactions for improved robustness