Beyond Visuals: Investigating Force Feedback in Extended Reality for Robot Data Collection

2025-03-26

AI Summary

This work investigates the impact of force feedback on robot data collection performance and user experience in extended reality (XR) environments—a problem尚未 systematically studied in robotics. We integrate haptic feedback into two XR teleoperation interfaces: kinesthetic teaching and motion-controller-based interaction. Through user studies involving multi-level precision tasks—including grasping, placing, and peg-in-hole assembly—we conduct the first systematic evaluation of force feedback’s role in XR-mediated robot demonstration learning. Results show that force feedback significantly improves task success rate (+23%) and subjective user satisfaction (+31%) for high-precision tasks; however, these gains are strongly modulated by interface type and task precision requirements. The study establishes force feedback’s critical role in acquiring high-fidelity demonstration data and provides empirical guidance and design principles for optimizing XR interfaces in robot learning systems.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This work explores how force feedback affects various aspects of robot data collection within the Extended Reality (XR) setting. Force feedback has been proved to enhance the user experience in Extended Reality (XR) by providing contact-rich information. However, its impact on robot data collection has not received much attention in the robotics community. This paper addresses this shortcoming by conducting an extensive user study on the effects of force feedback during data collection in XR. We extended two XR-based robot control interfaces, Kinesthetic Teaching and Motion Controllers, with haptic feedback features. The user study is conducted using manipulation tasks ranging from simple pick-place to complex peg assemble, requiring precise operations. The evaluations show that force feedback enhances task performance and user experience, particularly in tasks requiring high-precision manipulation. These improvements vary depending on the robot control interface and task complexity. This paper provides new insights into how different factors influence the impact of force feedback.

Problem

• Investigates force feedback's role in XR robot data collection
• Evaluates haptic effects on task performance and user experience
• Compares impact across control interfaces and task complexities

Innovation

• Extended force feedback in XR robot control
• Haptic-enhanced Kinesthetic Teaching and Motion Controllers
• Improved precision in manipulation tasks via feedback