ForceMimic: Force-Centric Imitation Learning with Force-Motion Capture System for Contact-Rich Manipulation

2024-10-10

arXiv.org

AI Summary

To address the insufficient learning of force-sensing skills in contact-intensive tasks (e.g., vegetable peeling), this paper proposes a force-motion coordinated imitation learning framework. We design ForceCapture—a robot-free, real-time force-feedback system enabling natural, high-fidelity synchronous capture of force and pose trajectories. We introduce the first force-centered imitation learning paradigm, constructing hybrid control primitives that jointly encode force and pose information, and propose HybridIL—an end-to-end training method parameterizing wrench and position jointly. Experiments on vegetable peeling demonstrate that our approach achieves a 54.5-percentage-point improvement in success rate over the state-of-the-art vision-only imitation learning (reaching 154.5% relative improvement), requires only five minutes per demonstration, and operates at 2.6× the efficiency of conventional teleoperation.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Motion & Tracking

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In most contact-rich manipulation tasks, humans apply time-varying forces to the target object, compensating for inaccuracies in the vision-guided hand trajectory. However, current robot learning algorithms primarily focus on trajectory-based policy, with limited attention given to learning force-related skills. To address this limitation, we introduce ForceMimic, a force-centric robot learning system, providing a natural, force-aware and robot-free robotic demonstration collection system, along with a hybrid force-motion imitation learning algorithm for robust contact-rich manipulation. Using the proposed ForceCapture system, an operator can peel a zucchini in 5 minutes, while force-feedback teleoperation takes over 13 minutes and struggles with task completion. With the collected data, we propose HybridIL to train a force-centric imitation learning model, equipped with hybrid force-position control primitive to fit the predicted wrench-position parameters during robot execution. Experiments demonstrate that our approach enables the model to learn a more robust policy under the contact-rich task of vegetable peeling, increasing the success rates by 54.5% relatively compared to state-ofthe-art pure-vision-based imitation learning. Hardware, code, data and more results can be found on the project website at https://forcemimic.github.io.

Problem

• Addresses lack of force-related skills in robot learning algorithms.
• Introduces ForceMimic for force-aware robotic demonstration collection.
• Improves success rates in contact-rich tasks like vegetable peeling.

Innovation

• ForceMimic system for force-centric robot learning
• HybridIL algorithm for hybrid force-position control
• ForceCapture system for efficient demonstration collection