GeCo-SRT: Geometry-aware Continual Adaptation for Robotic Cross-Task Sim-to-Real Transfer

2026-02-24

AI Summary

This work addresses the inefficiency of conventional simulation-to-reality (Sim2Real) transfer methods, which treat tasks in isolation, leading to redundant hyperparameter tuning and underutilized experience when adapting to new tasks. To overcome this limitation, the authors propose a continual cross-task Sim2Real transfer paradigm that, for the first time, incorporates a local geometric feature–based continual learning mechanism. This approach leverages a geometry-aware mixture-of-experts module combined with expert-guided prioritized experience replay to effectively mitigate catastrophic forgetting and reuse domain-invariant geometric knowledge. Experimental results demonstrate that the proposed method improves average performance by 52% over baseline approaches and achieves efficient adaptation to new tasks with only one-sixth of the training data, substantially enhancing both data efficiency and transfer robustness.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Bridging the sim-to-real gap is important for applying low-cost simulation data to real-world robotic systems. However, previous methods are severely limited by treating each transfer as an isolated endeavor, demanding repeated, costly tuning and wasting prior transfer experience.To move beyond isolated sim-to-real, we build a continual cross-task sim-to-real transfer paradigm centered on knowledge accumulation across iterative transfers, thereby enabling effective and efficient adaptation to novel tasks. Thus, we propose GeCo-SRT, a geometry-aware continual adaptation method. It utilizes domain-invariant and task-invariant knowledge from local geometric features as a transferable foundation to accelerate adaptation during subsequent sim-to-real transfers. This method starts with a geometry-aware mixture-of-experts module, which dynamically activates experts to specialize in distinct geometric knowledge to bridge observation sim-to-real gap. Further, the geometry-expert-guided prioritized experience replay module preferentially samples from underutilized experts, refreshing specialized knowledge to combat forgetting and maintain robust cross-task performance. Leveraging knowledge accumulated during iterative transfer, GeCo-SRT method not only achieves 52% average performance improvement over the baseline, but also demonstrates significant data efficiency for new task adaptation with only 1/6 data.We hope this work inspires approaches for efficient, low-cost cross-task sim-to-real transfer.

Problem

• sim-to-real transfer
• continual adaptation
• cross-task transfer
• knowledge accumulation
• robotic learning

Innovation

• continual adaptation
• sim-to-real transfer
• geometry-aware
• mixture-of-experts
• experience replay