A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies

2026-04-15

AI Summary

This work addresses the lack of theoretical understanding regarding the effectiveness of joint training with simulated and real data in generative robotic policies. Through theoretical modeling and multi-level experiments—including controlled toy models, simulation-to-simulation, and simulation-to-real robot manipulation tasks—the study uncovers two core mechanisms: structured representation alignment and action importance reweighting. It further elucidates the underlying principle governing the trade-off between cross-domain representation alignment and domain discriminability. Building on these insights, the authors propose a unified explanatory framework and an improved training method that consistently outperforms existing co-training strategies across diverse robotic manipulation tasks, thereby advancing both interpretability and performance in this domain.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Co-training, which combines limited in-domain real-world data with abundant surrogate data such as simulation or cross-embodiment robot data, is widely used for training generative robot policies. Despite its empirical success, the mechanisms that determine when and why co-training is effective remain poorly understood. We investigate the mechanism of sim-and-real co-training through theoretical analysis and empirical study, and identify two intrinsic effects governing performance. The first, \textbf{``structured representation alignment"}, reflects a balance between cross-domain representation alignment and domain discernibility, and plays a primary role in downstream performance. The second, the \textbf{``importance reweighting effect"}, arises from domain-dependent modulation of action weighting and operates at a secondary level. We validate these effects with controlled experiments on a toy model and extensive sim-and-sim and sim-and-real robot manipulation experiments. Our analysis offers a unified interpretation of recent co-training techniques and motivates a simple method that consistently improves upon prior approaches. More broadly, our aim is to examine the inner workings of co-training and to facilitate research in this direction.

Problem

• co-training
• sim-and-real
• generative robot policies
• representation alignment
• importance reweighting

Innovation

• structured representation alignment
• importance reweighting effect
• sim-and-real co-training
• generative robot policies
• domain adaptation