Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching

2026-05-10

AI Summary

Dexterous manipulation is highly sensitive to modeling errors and perceptual noise, posing significant challenges for sim-to-real transfer. To address this, this work proposes the Domain Randomization Instance Set (DRIS) method, which enables a policy to explicitly learn robustness against dynamic uncertainties by simultaneously sampling and propagating multiple randomized environment instances within a single training episode. The approach introduces an innovative multi-instance synchronous randomization mechanism, integrating reinforcement learning with theoretical analysis. Evaluated on a table tennis ball catching task without passive stabilization structures, DRIS achieves highly robust control and enables zero-shot sim-to-real transfer—requiring no fine-tuning in the real environment.

Abstract

Dexterous manipulation is physics-intensive and highly sensitive to modeling errors and perception noise, making sim-to-real transfer prohibitively challenging. Domain randomization (DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, we propose Domain-Randomized Instance Set (DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real-world fine-tuning, even with a modest number of instances (e.g., 10). We demonstrate this on a challenging reactive catching task. Unlike traditional catching setups that use end-effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), our system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero-shot sim-to-real transfer.

Problem

• zero-shot sim-to-real
• dexterous manipulation
• reactive catching
• modeling errors
• perception noise

Innovation

• Domain-Randomized Instance Set
• zero-shot sim-to-real
• dexterous manipulation
• reactive catching
• robust policy learning