A Closed-Form Geometric Retargeting Solver for Upper Body Humanoid Robot Teleoperation

2026-02-02

AI Summary

Existing teleoperation methods for humanoid robot upper limbs commonly suffer from computational latency, suboptimal solutions, and limited workspace. This work addresses these challenges by formulating human-to-robot upper-limb motion retargeting as a shoulder–elbow–wrist (SEW) keypoint-driven alignment of upper and lower arm orientations. We propose SEW-Mimic, a closed-form geometric solver with optimality guarantees that accepts generic keypoint inputs and integrates a safety filter to prevent self-collisions. Implemented on a standard CPU, the method achieves real-time inference at 3 kHz, substantially outperforming existing approaches in both accuracy and efficiency. User studies demonstrate that SEW-Mimic improves task success rates, produces smoother teleoperation trajectories, and effectively accelerates full-body retargeting and hardware deployment.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Retargeting human motion to robot poses is a practical approach for teleoperating bimanual humanoid robot arms, but existing methods can be suboptimal and slow, often causing undesirable motion or latency. This is due to optimizing to match robot end-effector to human hand position and orientation, which can also limit the robot's workspace to that of the human. Instead, this paper reframes retargeting as an orientation alignment problem, enabling a closed-form, geometric solution algorithm with an optimality guarantee. The key idea is to align a robot arm to a human's upper and lower arm orientations, as identified from shoulder, elbow, and wrist (SEW) keypoints; hence, the method is called SEW-Mimic. The method has fast inference (3 kHz) on standard commercial CPUs, leaving computational overhead for downstream applications; an example in this paper is a safety filter to avoid bimanual self-collision. The method suits most 7-degree-of-freedom robot arms and humanoids, and is agnostic to input keypoint source. Experiments show that SEW-Mimic outperforms other retargeting methods in computation time and accuracy. A pilot user study suggests that the method improves teleoperation task success. Preliminary analysis indicates that data collected with SEW-Mimic improves policy learning due to being smoother. SEW-Mimic is also shown to be a drop-in way to accelerate full-body humanoid retargeting. Finally, hardware demonstrations illustrate SEW-Mimic's practicality. The results emphasize the utility of SEW-Mimic as a fundamental building block for bimanual robot manipulation and humanoid robot teleoperation.

Problem

• retargeting
• humanoid robot
• teleoperation
• motion mapping
• bimanual manipulation

Innovation

• closed-form solver
• geometric retargeting
• orientation alignment
• SEW-Mimic
• humanoid teleoperation