ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning

2026-08-19

AI Summary

研究通过预训练和后训练方法，使用强化学习框架ADEPT解决高自由度机器人从模拟到现实的灵巧技能迁移问题。

Abstract

We introduce Accelerating Dexterity via Pre-Training (ADEPT), a large-scale reinforcement learning (RL) framework for learning sim-to-real transferable dexterity across high degree-of-freedom (DoF) robot embodiments that can solve long-horizon tasks directly from raw visuo-tactile perception. ADEPT pretrains a dexterous policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior. ADEPT enables learning new behaviors that are otherwise difficult to discover from scratch on multi-fingered robots and avoids learning the same set of skills over again for every new downstream task. The pretrained policy zero-shots the reposing phase of downstream tasks, but naïve RL fine-tuning rapidly degrades this capability during transfer. We address this with a stable post-training recipe combining behavior-cloning distillation, critic warm-up, and conservative on-policy updates. To safely exploit the full kinematic dexterity, we introduce a joint-space Geometric Fabric that mediates between the RL policy and the robot. We distill post-trained teachers into perceptive students that zero-shot sim-to-real transfer on two embodiments: a 23 DoF Kuka-Allegro with two RGB cameras, and a 29 DoF Flexiv-Sharpa with two RGB cameras and five vision-based tactile sensors, and can solve long-horizon tasks from challenging initial states with dexterity at human-level speed.

Problem

• Reinforcement Learning
• Sim-to-Real Transfer
• Dexterity
• High Degree-of-Freedom Robots
• Visuo-Tactile Perception

Innovation

• reinforcement learning
• pre-training and post-training
• sim-to-real transfer
• high degree-of-freedom robots
• behavior-cloning distillation