Object-Focus Actor for Data-efficient Robot Generalization Dexterous Manipulation

2025-05-21

AI Summary

Weak generalization across diverse scenes and object poses, coupled with high demonstration requirements, hinders robotic dexterous manipulation. This paper proposes an object-centric hierarchical policy framework featuring a novel end-effector trajectory consistency–based object focusing mechanism, enabling strong generalization from only ten human demonstrations. The method integrates a three-stage vision–action co-design: (1) object perception and 6D pose estimation; (2) pre-manipulation pose reaching planning; and (3) a lightweight Object-Focus Actor policy network. Evaluated on seven real-world dexterous manipulation tasks, the approach significantly improves both positional and background generalization, achieving robust and transferable manipulation performance with minimal demonstration cost.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Robot manipulation learning from human demonstrations offers a rapid means to acquire skills but often lacks generalization across diverse scenes and object placements. This limitation hinders real-world applications, particularly in complex tasks requiring dexterous manipulation. Vision-Language-Action (VLA) paradigm leverages large-scale data to enhance generalization. However, due to data scarcity, VLA's performance remains limited. In this work, we introduce Object-Focus Actor (OFA), a novel, data-efficient approach for generalized dexterous manipulation. OFA exploits the consistent end trajectories observed in dexterous manipulation tasks, allowing for efficient policy training. Our method employs a hierarchical pipeline: object perception and pose estimation, pre-manipulation pose arrival and OFA policy execution. This process ensures that the manipulation is focused and efficient, even in varied backgrounds and positional layout. Comprehensive real-world experiments across seven tasks demonstrate that OFA significantly outperforms baseline methods in both positional and background generalization tests. Notably, OFA achieves robust performance with only 10 demonstrations, highlighting its data efficiency.

Problem

• Lack of generalization in robot manipulation learning from human demonstrations
• Data scarcity limits Vision-Language-Action (VLA) paradigm performance
• Need for data-efficient dexterous manipulation across diverse scenes

Innovation

• Object-Focus Actor for efficient policy training
• Hierarchical pipeline with object perception
• Robust performance with minimal demonstrations