teleoperation interface design

This study addresses the critical challenges of low-quality teleoperation data and poor device-controller compatibility, which hinder the training of embodied intelligence foundation models. To this end, we propose a unified teleoperation data collection framework. Methodologically, we design a quadratic programming (QP)-based optimal controller that integrates dynamic null-space projection and impedance tracking, with adaptive weight tuning to enable joint manipulability-aware compliant control and singularity avoidance. Our end-to-end pipeline unifies position-based inverse kinematics, torque-based inverse dynamics, optimization-based admittance control, and the QP framework. Extensive experiments across diverse robot-device–controller configurations quantitatively evaluate trajectory tracking error, singularity occurrence rate, and joint motion smoothness. Results demonstrate substantial improvements in data stability and diversity, yielding a high-quality, broad-coverage robotic skill dataset essential for advancing embodied intelligence.

To address key challenges in human–machine interfaces (HMIs) for underwater remotely operated vehicles—including low immersion, unintuitive control, lack of cross-platform standards, and insufficient shared autonomy—this study systematically reviews over 100 state-of-the-art approaches, identifying turbid-water perception degradation and communication latency as fundamental bottlenecks. We propose a novel multidimensional HMI evaluation framework integrating human-centered metrics (e.g., intuitiveness, situational awareness) with robotic performance criteria (safety, accuracy, efficiency). Furthermore, we introduce a bidirectional intelligent collaboration paradigm supported by multimodal feedback, incorporating real-time gesture recognition, VR/AR-based rendering, large language model–driven natural language interaction, ultra-low-latency communication, and multi-source underwater state estimation. Finally, we distill six open research questions and outline an interdisciplinary roadmap, providing theoretical foundations and design guidelines for next-generation intelligent deep-sea intervention systems.

Novice teleoperators often produce demonstration data that, while task-successful, exhibit poor quality—such as inefficient motions, frequent corrections, or proximity to joint limits—rendering them inadequate for downstream learning. To address this, this work proposes the Data Quality Assessment and Feedback (DQAF) framework, which integrates multimodal signals including subtask progress, motion smoothness, stalling behavior, and joint limit proximity to generate, after each operation, a structured quality score accompanied by interpretable natural language feedback. Moving beyond conventional binary success/failure judgments, DQAF enables the first fine-grained quality evaluation and closed-loop guidance tailored specifically to teleoperation. Experimental results demonstrate strong alignment between system-generated feedback and human expert assessments, and show that novices receiving such feedback significantly improve both the quality and efficiency of their subsequent demonstrations.

To address the challenge of enabling non-expert users to efficiently operate humanoid robots in FIRA-standard obstacle courses, this paper designs and implements a lightweight, event-driven graphical user interface (GUI). Grounded in human-robot interaction (HRI) theory, the GUI adopts a simplified operational paradigm and multimodal visual feedback mechanisms to significantly reduce the cognitive load and learning curve. It supports remote teleoperation—enabling intuitive path planning, action triggering, and real-time robot state monitoring without specialized training. Experimental evaluation demonstrates a 42% improvement in task completion rate and a 35% reduction in average operation time among non-expert users. The primary contributions are: (1) a beginner-oriented GUI design framework tailored for standardized robotic competitions, and (2) empirical validation of the effectiveness and scalability of low-cognitive-load teleoperation interfaces in such contexts.

Novice operators face significant challenges in teleoperating robots, including high operational complexity, poor safety guarantees, and limited cross-platform compatibility—factors that hinder both robot learning and efficient data collection. To address these issues, we propose a low-cost (<$1,000) teleoperation system featuring a novel real-time virtual arm visualization mechanism that enables pre-execution rehearsal of commands, supporting seamless toggling between action preview and execution. The system leverages lightweight motion mapping, real-time virtual rendering, and low-latency interactive feedback, requiring no specialized hardware and ensuring compatibility with mainstream robotic arm platforms. Evaluated on five dexterous manipulation tasks, our approach outperforms existing methods in task success rate and operator efficiency. It substantially reduces the learning curve for novice users and improves operational safety. All source code and deployment documentation are publicly released under an open-source license.

Existing teleoperation systems are often tailored to specific hardware and tasks, lacking generality and scalability. This work proposes ModPack—a modular and extensible teleoperation framework that leverages a wearable backpack platform integrating computation, power, communication, and storage functionalities, coupled with a unified wearable interface architecture. This design enables plug-and-play integration of capability modules such as joint-level teleoperation, mobile manipulation, and active perception. The system’s efficacy in real-world mobile manipulation tasks is validated across two heterogeneous robotic platforms, demonstrating significantly enhanced reusability and adaptability. To foster community advancement, the authors open-source all hardware and software designs.

This study addresses the challenges of high cognitive load on single operators and substantial coordination overhead in multi-operator setups for multi-arm teleoperation tasks. The authors propose a human–robot collaborative teleoperation framework wherein a human directly controls two master arms, while two auxiliary arms are autonomously managed by a training-free multimodal large language model (MLLM) agent to execute subtasks, with real-time intervention enabled via voice commands. This approach pioneers the integration of MLLM agents into multi-arm teleoperation for data collection, achieving decoupled control spaces and natural human–robot interaction. The system maintains high operational efficiency while significantly enhancing scalability. Experimental results demonstrate that the proposed method achieves data collection success rates and efficiency comparable to those of expert two-human teams, and the collected data effectively supports downstream training of multi-arm collaborative policies.

This work addresses the challenges of low arm–hand coordination accuracy, high feedback latency, and unstable contact interaction in dexterous teleoperation by proposing a modular bilateral teleoperation system that seamlessly integrates operator inputs with a compliant robotic arm and a dexterous hand. The system employs key techniques—including position retargeting, differential arm control, multi-scale tactile feedback, and shared autonomy—to establish a coordinated control architecture tailored for real-world contact-rich environments. It further elucidates design principles concerning cross-embodiment mismatch, tactile feedback granularity, and shared control strategies. Experimental results demonstrate that the system achieves stable and efficient arm–hand coordination in complex dexterous manipulation tasks and provides a high-quality data acquisition platform for imitation learning.

This work addresses a critical limitation in current robotic AI systems, which often prioritize surpassing human performance in single-task scenarios while lacking the capacity to detect and recover from errors during continuous human–robot interaction. Focusing on nuclear glovebox operations as a high-stakes application domain, the study proposes a novel robotic system architecture explicitly designed around error recovery. Integrating principles from human factors engineering and interactive AI, the framework incorporates mechanisms for real-time error detection, feedback loops, and adaptive adjustment to enhance robustness and adaptability in dynamic, real-world environments. By centering system design on resilience and learning from mistakes, this approach establishes a new paradigm for fault-tolerant and adaptive human–robot collaboration in safety-critical settings.

Existing teleoperation methods often fail in dexterous grasping of dynamically moving objects due to errors in timing, pose estimation, and force control. This work proposes Tele-Catch, a framework that integrates human teleoperation signals with autonomous policies to achieve robust grasping of dynamic 3D objects. The key innovations include a Dynamics-Aware Adaptive Shared Control mechanism (DAIM) that dynamically adjusts the control authority between human and robot based on real-time object states, and a diffusion-policy-based denoising grasp generation module coupled with DP-U3R—an unsupervised point cloud geometric representation method—to enhance policy generalization. Experimental results demonstrate that Tele-Catch significantly improves both success rate and robustness across multiple dexterous hand platforms and unseen object categories.