Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training

2025-09-23

AI Summary

To address the scarcity of real-world demonstrations and the Sim2Real transfer challenge in robotic manipulation, this paper proposes a unified framework for co-training on simulated and real data. Methodologically, it integrates behavioral cloning with optimal transport theory, jointly optimizing the policy under feature-space alignment. Its key contributions are: (1) constructing a domain-invariant, task-relevant feature space; (2) introducing an observation-action joint distribution alignment mechanism to enhance cross-domain consistency; and (3) adopting an unbalanced optimal transport formulation to mitigate data-scale mismatch between simulation and reality. Experiments demonstrate that the method achieves up to a 30% absolute improvement in real-world task success rates using fewer than 50 real demonstrations, while also generalizing effectively to novel scenes unseen during simulation training.

Technology Category

• Intelligent Robots: Manipulation
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Behavior cloning has shown promise for robot manipulation, but real-world demonstrations are costly to acquire at scale. While simulated data offers a scalable alternative, particularly with advances in automated demonstration generation, transferring policies to the real world is hampered by various simulation and real domain gaps. In this work, we propose a unified sim-and-real co-training framework for learning generalizable manipulation policies that primarily leverages simulation and only requires a few real-world demonstrations. Central to our approach is learning a domain-invariant, task-relevant feature space. Our key insight is that aligning the joint distributions of observations and their corresponding actions across domains provides a richer signal than aligning observations (marginals) alone. We achieve this by embedding an Optimal Transport (OT)-inspired loss within the co-training framework, and extend this to an Unbalanced OT framework to handle the imbalance between abundant simulation data and limited real-world examples. We validate our method on challenging manipulation tasks, showing it can leverage abundant simulation data to achieve up to a 30% improvement in the real-world success rate and even generalize to scenarios seen only in simulation.

Problem

• Addressing costly real-world robot demonstration acquisition at scale
• Overcoming simulation-to-real domain gaps for policy transfer
• Learning domain-invariant policies with limited real-world examples

Innovation

• Sim-and-real co-training with few real demos
• Domain-invariant features via joint distribution alignment
• Unbalanced Optimal Transport for data imbalance