Mastering Contact-rich Tasks by Combining Soft and Rigid Robotics with Imitation Learning

2024-10-10

arXiv.org

AI Summary

To address the trade-off between low control precision in soft robots and poor environmental adaptability in rigid robots, this work proposes a rigid–soft coupled robotic system and a cross-modal imitation learning framework. Hardware-wise, we introduce a fully integrated rigid–soft co-design architecture that synergistically combines the high-precision positioning capability of rigid manipulators with the safe, adaptive physical interaction enabled by soft actuators. Algorithmically, we develop a multimodal imitation learning method tailored for contact-intensive tasks, enabling end-to-end joint modeling of tactile, visual, and proprioceptive motion signals. Evaluated on grasping, assembly, and compliant manipulation tasks, our system achieves success rates exceeding 92%; for unseen objects and environments, task generalization accuracy reaches 86%, significantly outperforming both purely rigid and purely soft baselines. The core contributions are: (i) a paradigm of deep hardware integration of rigid and soft components, and (ii) a learning framework that jointly optimizes safety, precision, and generalizability.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Soft robots have the potential to revolutionize the use of robotic systems with their capability of establishing safe, robust, and adaptable interactions with their environment, but their precise control remains challenging. In contrast, traditional rigid robots offer high accuracy and repeatability but lack the flexibility of soft robots. We argue that combining these characteristics in a hybrid robotic platform can significantly enhance overall capabilities. This work presents a novel hybrid robotic platform that integrates a rigid manipulator with a fully developed soft arm. This system is equipped with the intelligence necessary to perform flexible and generalizable tasks through imitation learning autonomously. The physical softness and machine learning enable our platform to achieve highly generalizable skills, while the rigid components ensure precision and repeatability.

Problem

• Combining soft and rigid robotics for contact-rich tasks
• Overcoming precise control challenges in soft robotics
• Enhancing robotic capabilities through hybrid systems and imitation learning

Innovation

• Hybrid platform combining rigid and soft robotics
• Imitation learning for autonomous task generalization
• Soft arm ensures adaptability while rigid parts provide precision