Goal-Conditioned Dual-Action Imitation Learning for Dexterous Dual-Arm Robot Manipulation

2022-03-18

IEEE Transactions on robotics

AI Summary

Long-horizon dexterous bimanual manipulation of deformable objects (e.g., bananas) remains challenging due to difficulties in physical modeling, poor skill generalization, and instability caused by error accumulation. Method: We propose a target-conditioned dual-action imitation learning framework featuring a novel synergistic mechanism: “local reactive actions” driven by real-time sensor feedback to rapidly respond to sudden deformations, and “global trajectory generation” via a target-conditioned neural network to suppress error accumulation—integrated with bimanual kinematic modeling and deep imitation learning. Contribution/Results: Evaluated on a real bimanual robotic platform, our method achieves fully autonomous banana peeling. It improves task success rate by 42% and significantly enhances operational stability, demonstrating strong efficacy and generalizability for complex non-rigid manipulation tasks.

Abstract

Long-horizon dexterous robot manipulation of deformable objects, such as banana peeling, is a problematic task because of the difficulties in object modeling and a lack of knowledge about stable and dexterous manipulation skills. This article presents a goal-conditioned dual-action deep imitation learning (DIL) approach that can learn dexterous manipulation skills using human demonstration data. Previous DIL methods map the current sensory input and reactive action, which often fails because of compounding errors in imitation learning caused by the recurrent computation of actions. The method predicts reactive action only when the precise manipulation of the target object is required (local action) and generates the entire trajectory when precise manipulation is not required (global action). This dual-action formulation effectively prevents compounding error in the imitation learning using the trajectory-based global action while responding to unexpected changes in the target object during the reactive local action. The proposed method was tested in a real dual-arm robot and successfully accomplished the banana-peeling task.

Problem

• Long-horizon dexterous manipulation of deformable objects
• Compounding errors in imitation learning methods
• Dual-action formulation for precise and global manipulation

Innovation

• Goal-conditioned dual-action imitation learning
• Combines local and global action prediction
• Reduces compounding errors in imitation learning