MR-UBi: Mixed Reality-Based Underwater Robot Arm Teleoperation System with Reaction Torque Indicator via Bilateral Control

2025-10-23

AI Summary

In underwater robotic arm teleoperation, grasping deformable objects suffers from low torque control accuracy, frequent torque limit violations, and weak operator awareness. To address these challenges, this paper proposes a teleoperation system integrating mixed reality (MR) with bidirectional haptic feedback. Its core innovations include: (1) a reaction torque indicator encoded by both color and length, rendered in real time within an MR headset to visualize joint torque; and (2) a vision–haptics synergistic bidirectional control framework incorporating high-precision force/position sensing and a human–robot collaborative interface. Experimental results demonstrate that the system significantly improves grasp force control accuracy (32.7% reduction in error), extends the duration of optimal torque application (+41.5%), reduces operator workload by 28.3%, and increases usability scores by 37.1%. Overall, it enhances operational stability and user experience in underwater manipulation tasks.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Interaction Techniques and Devices
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a mixed reality-based underwater robot arm teleoperation system with a reaction torque indicator via bilateral control (MR-UBi). The reaction torque indicator (RTI) overlays a color and length-coded torque bar in the MR-HMD, enabling seamless integration of visual and haptic feedback during underwater robot arm teleoperation. User studies with sixteen participants compared MR-UBi against a bilateral-control baseline. MR-UBi significantly improved grasping-torque control accuracy, increasing the time within the optimal torque range and reducing both low and high grasping torque range during lift and pick-and-place tasks with objects of different stiffness. Subjective evaluations further showed higher usability (SUS) and lower workload (NASA--TLX). Overall, the results confirm that extit{MR-UBi} enables more stable, accurate, and user-friendly underwater robot-arm teleoperation through the integration of visual and haptic feedback. For additional material, please check: https://mertcookimg.github.io/mr-ubi

Problem

• Improving underwater robot arm teleoperation accuracy through mixed reality
• Integrating visual and haptic feedback for stable grasping torque control
• Enhancing user experience with reduced workload in teleoperation tasks

Innovation

• Mixed reality system for underwater robot teleoperation
• Bilateral control with reaction torque indicator
• Integrates visual and haptic feedback for accuracy