FAR-Dex: Few-shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulation

2026-03-11

AI Summary

This work addresses the challenges of scarce high-quality demonstration data and complex high-dimensional action spaces in coordinated manipulation tasks involving multi-fingered dexterous hands and robotic arms. The authors propose FAR-Dex, a hierarchical framework that uniquely integrates few-shot trajectory augmentation with an adaptive residual policy optimization based on multi-step trajectory segments. Leveraging the IsaacLab simulator, the method generates physically plausible trajectories and constructs a policy network that fuses observational features with trajectory segments, significantly enhancing positional generalization. Experimental results demonstrate that the approach improves task success rates by 7% and data quality by 13.4% in both simulation and real-world settings, achieving a real-world task success rate exceeding 80%.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Achieving human-like dexterous manipulation through the collaboration of multi-fingered hands with robotic arms remains a longstanding challenge in robotics, primarily due to the scarcity of high-quality demonstrations and the complexity of high-dimensional action spaces. To address these challenges, we propose FAR-Dex, a hierarchical framework that integrates few-shot data augmentation with adaptive residual refinement to enable robust and precise arm-hand coordination in dexterous tasks. First, FAR-DexGen leverages the IsaacLab simulator to generate diverse and physically constrained trajectories from a few demonstrations, providing a data foundation for policy training. Second, FAR-DexRes introduces an adaptive residual module that refines policies by combining multi-step trajectory segments with observation features, thereby enhancing accuracy and robustness in manipulation scenarios. Experiments in both simulation and real-world demonstrate that FAR-Dex improves data quality by 13.4% and task success rates by 7% over state-of-the-art methods. It further achieves over 80% success in real-world tasks, enabling fine-grained dexterous manipulation with strong positional generalization.

Problem

• dexterous manipulation
• few-shot learning
• high-dimensional action spaces
• data scarcity
• arm-hand coordination

Innovation

• few-shot data augmentation
• adaptive residual policy refinement
• dexterous manipulation
• hierarchical framework
• trajectory generation