MIRROR: Visual Motion Imitation via Real-time Retargeting and Teleoperation with Parallel Differential Inverse Kinematics

2026-03-25

AI Summary

This work addresses the challenges of real-time humanoid robot teleoperation, where conventional differential inverse kinematics (IK) often suffers from joint limits, singularities, and self-collision constraints, leading to local minima and compromising responsiveness, safety, and stability. To overcome these limitations, we propose a GPU-accelerated, continuously optimized differential IK framework that simultaneously solves multiple constrained quadratic programs. The approach integrates Control Barrier Functions (CBFs) to enforce self-collision avoidance and incorporates a Lyapunov-based progress condition to guarantee global descent of task-space tracking errors. Evaluated on the THEMIS humanoid platform, our method demonstrates significantly enhanced capability to escape local minima while enabling robust, real-time upper-body teleoperation with strong obstacle avoidance and numerical stability.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Real-time humanoid teleoperation requires inverse kinematics (IK) solvers that are both responsive and constraint-safe under kinematic redundancy and self-collision constraints. While differential IK enables efficient online retargeting, its locally linearized updates are inherently basin-dependent and often become trapped near joint limits, singularities, or active collision boundaries, leading to unsafe or stagnant behavior. We propose a GPU-parallelized, continuation-based differential IK that improves escape from such constraint-induced local minima while preserving real-time performance, promoting safety and stability. Multiple constrained IK quadratic programs are evaluated in parallel, together with a self-collision avoidance control barrier function (CBF), and a Lyapunov-based progression criterion selects updates that reduce the final global task-space error. The method is paired with a visual skeletal pose estimation pipeline that enables robust, real-time upper-body teleoperation on the THEMIS humanoid robot hardware in real-world tasks.

Problem

• inverse kinematics
• real-time teleoperation
• constraint handling
• local minima
• humanoid robot

Innovation

• differential inverse kinematics
• GPU parallelization
• control barrier function
• real-time teleoperation
• constraint handling