Gated Residual Body-Hand Coordination for Whole-Body Humanoid Teleoperation

2026-09-16

AI Summary

本文提出一种门控残差协调框架，通过修正独立生成的命令来保持身体与手部几何关系的一致性，从而解决全身体感操作中手部与身体动作不匹配的问题。

Abstract

Whole-body humanoid teleoperation commonly combines a motion-tracking policy with a separate dexterous-hand retargeter. However, independently generated commands do not explicitly preserve body-hand geometric relations, leading to mismatches in relative wrist poses and fingertip positions during bimanual interaction. We present a gated residual coordination framework that keeps both modules frozen and applies bounded corrections to their outputs. A motion-conditioned action gate allocates correction authority across joint groups, while reference-geometry-dependent reward gates emphasize relevant interaction objectives during training. To establish the nominal body controller on Agile One, we introduce multi-pose morphology calibration that jointly estimates triaxial scales and effector-local offsets, together with staged motion dataset curation for training a SONIC-based tracker. The residual policy uses human motion references, initial commands, and robot proprioception without explicit object or contact observations. In simulation, it reduces wrist and fingertip geometry errors by 39.2-56.3% over direct composition on held-out GRAB motions, while preserving whole-body tracking on AMASS, with success rates of 89.03% without residual coordination and 89.29% with it. Ablations characterize the contributions of reward gating, adaptive correction authority, and separate body and hand correction heads.

Problem

• Whole-body humanoid teleoperation
• motion-tracking policy
• dexterous-hand retargeter
• body-hand geometric relations
• mismatches

Innovation

• gated residual coordination
• motion-conditioned action gate
• reference-geometry-dependent reward gates
• multi-pose morphology calibration