Information-Theoretic Detection of Bimanual Interactions for Dual-Arm Robot Plan Generation
🤖 AI Summary
This work addresses the challenge of efficiently generating executable plans for dual-arm robotic tasks from human demonstrations, where bimanual coordination strategies are often complex and difficult to model. The authors propose a novel approach that leverages a single RGB video demonstration to synthesize structured, modular behavior tree plans. Their method uniquely integrates Shannon information theory to analyze information flow between hands, scene graph parsing to extract action semantics, and one-shot learning to enable generalization. By unifying these components within a behavior tree framework, the approach produces adaptable execution plans without requiring extensive training data. Evaluated on both a newly curated dataset and existing public benchmarks, the method demonstrates significant performance gains over current state-of-the-art techniques, marking a notable advance in centralized bimanual coordination planning.
Application Category
📝 Abstract
Programming by demonstration is a strategy to simplify the robot programming process for non-experts via human demonstrations. However, its adoption for bimanual tasks is an underexplored problem due to the complexity of hand coordination, which also hinders data recording. This letter presents a novel one-shot method for processing a single RGB video of a bimanual task demonstration to generate an execution plan for a dual-arm robotic system. To detect hand coordination policies, we apply Shannon's information theory to analyze the information flow between scene elements and leverage scene graph properties. The generated plan is a modular behavior tree that assumes different structures based on the desired arms coordination. We validated the effectiveness of this framework through multiple subject video demonstrations, which we collected and made open-source, and exploiting data from an external, publicly available dataset. Comparisons with existing methods revealed significant improvements in generating a centralized execution plan for coordinating two-arm systems.
Problem
Research questions and friction points this paper is trying to address.
bimanual interaction
programming by demonstration
dual-arm robot
hand coordination
execution plan generation
Innovation
Methods, ideas, or system contributions that make the work stand out.
information theory
bimanual coordination
programming by demonstration
behavior tree
scene graph
🔎 Similar Papers
No similar papers found.
