DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation

2025-09-28

AI Summary

Tactile manipulation has long suffered from a scarcity of high-quality, diverse datasets, resulting in significant scalability and generalization bottlenecks for existing methods. To address this, we propose a self-iterative data flywheel framework: starting from a small set of seed demonstrations, it integrates imitation learning, residual reinforcement learning, simulated trajectory rollouts, and cross-environment data augmentation into a closed-loop data generation pipeline—jointly enhancing both data diversity and policy performance. The framework autonomously generates over 2,000 high-diversity demonstrations. Evaluated on four challenging simulated tasks, it achieves an average success rate of 81.9%, and successfully transfers to a real-world dual-arm lifting task with a 78.3% success rate. This demonstrates substantial improvements in policy generalization and robustness for real-world deployment.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Manipulation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Dexterous manipulation is critical for advancing robot capabilities in real-world applications, yet diverse and high-quality datasets remain scarce. Existing data collection methods either rely on human teleoperation or require significant human engineering, or generate data with limited diversity, which restricts their scalability and generalization. In this paper, we introduce DexFlyWheel, a scalable data generation framework that employs a self-improving cycle to continuously enrich data diversity. Starting from efficient seed demonstrations warmup, DexFlyWheel expands the dataset through iterative cycles. Each cycle follows a closed-loop pipeline that integrates Imitation Learning (IL), residual Reinforcement Learning (RL), rollout trajectory collection, and data augmentation. Specifically, IL extracts human-like behaviors from demonstrations, and residual RL enhances policy generalization. The learned policy is then used to generate trajectories in simulation, which are further augmented across diverse environments and spatial configurations before being fed back into the next cycle. Over successive iterations, a self-improving data flywheel effect emerges, producing datasets that cover diverse scenarios and thereby scaling policy performance. Experimental results demonstrate that DexFlyWheel generates over 2,000 diverse demonstrations across four challenging tasks. Policies trained on our dataset achieve an average success rate of 81.9% on the challenge test sets and successfully transfer to the real world through digital twin, achieving a 78.3% success rate on dual-arm lift tasks.

Problem

• Scarcity of diverse high-quality datasets for dexterous robot manipulation
• Limitations in scalability and generalization of existing data collection methods
• Need for automated framework to continuously enrich manipulation data diversity

Innovation

• Self-improving cycle for continuous data enrichment
• Closed-loop pipeline integrating IL and residual RL
• Data augmentation across diverse environments and configurations