DeRi-IGP: Learning to Manipulate Rigid Objects Using Deformable Objects via Iterative Grasp-Pull

2023-09-09

AI Summary

To address limitations in robotic manipulation—including constrained action spaces, poor generalization, and high model development costs—this paper introduces a novel paradigm: manipulating rigid objects via deformable linear objects (DLOs), such as ropes. Our method integrates DLO dynamics modeling, simulation-to-real transfer training, and multi-modal sensory fusion. Key contributions are: (1) the Iterative Grasp-and-Pull (IGP) motion primitive, the first unified representation for soft-rigid coupled manipulation driven by DLOs; (2) an end-to-end vision-based neural policy enabling parameterized closed-loop DLO control; and (3) a decentralized multi-agent coordination and human-robot collaboration framework. Evaluated in both simulation and real-world setups, our approach achieves high task success rates. In human-robot collaborative transport tasks, it attains >92% completion rate—significantly outperforming state-of-the-art model-driven and purely learning-based baselines.

Abstract

Robotic manipulation of rigid objects via deformable linear objects (DLO) such as ropes is an emerging field of research with applications in various rigid object transportation tasks. A few methods that exist in this field suffer from limited robot action and operational space, poor generalization ability, and expensive model-based development. To address these challenges, we propose a universally applicable moving primitive called Iterative Grasp-Pull (IGP). We also introduce a novel vision-based neural policy that learns to parameterize the IGP primitive to manipulate DLO and transport their attached rigid objects to the desired goal locations. Additionally, our decentralized algorithm design allows collaboration among multiple agents to manipulate rigid objects using DLO. We evaluated the effectiveness of our approach in both simulated and real-world environments for a variety of soft-rigid body manipulation tasks. In the real world, we also demonstrate the effectiveness of our decentralized approach through human-robot collaborative transportation of rigid objects to given goal locations. We also showcase the large operational space of IGP primitive by solving distant object acquisition tasks. Lastly, we compared our approach with several model-based and learning-based baseline methods. The results indicate that our method surpasses other approaches by a significant margin.

Problem

• Manipulate rigid objects via deformable linear objects
• Improve robot action and operational space
• Enable decentralized multi-agent collaboration

Innovation

• Iterative Grasp-Pull primitive
• Vision-based neural policy
• Decentralized multi-agent collaboration