Glovity: Learning Dexterous Contact-Rich Manipulation via Spatial Wrench Feedback Teleoperation System

2025-10-10

AI Summary

Addressing the dual challenges of absent haptic feedback and motion redirection discrepancies in rich-contact teleoperation, this paper proposes a low-cost wearable teleoperation system integrating spatial torque feedback and tactile perception. The system combines six-axis torque sensors, a Hall-effect-based fingertip pose calibration module, and vibrotactile gloves, augmented by a kinematic motion redirection algorithm and an enhanced DP-R3M imitation learning framework. Crucially, it is the first to explicitly incorporate real-time force signals into policy learning, enabling novel tasks such as adaptive page-turning and force-aware object handover. User studies demonstrate that force feedback improves page-turning success rate by 30 percentage points (from 48% to 78%) and reduces task completion time by 25%; fingertip pose calibration significantly enhances stable grasping of thin objects. All hardware designs and software implementations are fully open-sourced.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Interaction Techniques and Devices
• Computer Vision: Motion & Tracking

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We present Glovity, a novel, low-cost wearable teleoperation system that integrates a spatial wrench (force-torque) feedback device with a haptic glove featuring fingertip Hall sensor calibration, enabling feedback-rich dexterous manipulation. Glovity addresses key challenges in contact-rich tasks by providing intuitive wrench and tactile feedback, while overcoming embodiment gaps through precise retargeting. User studies demonstrate significant improvements: wrench feedback boosts success rates in book-flipping tasks from 48% to 78% and reduces completion time by 25%, while fingertip calibration enhances thin-object grasping success significantly compared to commercial glove. Furthermore, incorporating wrench signals into imitation learning (via DP-R3M) achieves high success rate in novel contact-rich scenarios, such as adaptive page flipping and force-aware handovers. All hardware designs, software will be open-sourced. Project website: https://glovity.github.io/

Problem

• Developing a low-cost teleoperation system for dexterous contact-rich manipulation tasks
• Providing intuitive wrench and tactile feedback to overcome embodiment gaps
• Enhancing imitation learning performance in novel contact-rich scenarios

Innovation

• Low-cost wearable teleoperation with spatial wrench feedback
• Fingertip Hall sensor calibration for haptic glove
• Wrench signals integration into imitation learning framework