CLOT: Closed-Loop Global Motion Tracking for Whole-Body Humanoid Teleoperation

2026-02-13

AI Summary

This work proposes a closed-loop global motion tracking system to address instability caused by global pose drift during prolonged full-body teleoperation. By integrating high-frequency localization feedback with a Transformer-based reinforcement learning policy, the system decouples observed trajectories from reward evaluation through a data-driven randomization mechanism. An adversarial motion prior is further introduced to suppress unnatural movements, enabling smooth and drift-free pose synchronization. Requiring only 20 hours of human motion data, the method achieves high-dynamic, high-precision, and robust sim-to-real transfer on a full-scale humanoid robot with 31 degrees of freedom, supporting stable long-duration teleoperation.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Motion and Path Planning
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Long-horizon whole-body humanoid teleoperation remains challenging due to accumulated global pose drift, particularly on full-sized humanoids. Although recent learning-based tracking methods enable agile and coordinated motions, they typically operate in the robot's local frame and neglect global pose feedback, leading to drift and instability during extended execution. In this work, we present CLOT, a real-time whole-body humanoid teleoperation system that achieves closed-loop global motion tracking via high-frequency localization feedback. CLOT synchronizes operator and robot poses in a closed loop, enabling drift-free human-to-humanoid mimicry over long timehorizons. However, directly imposing global tracking rewards in reinforcement learning, often results in aggressive and brittle corrections. To address this, we propose a data-driven randomization strategy that decouples observation trajectories from reward evaluation, enabling smooth and stable global corrections. We further regularize the policy with an adversarial motion prior to suppress unnatural behaviors. To support CLOT, we collect 20 hours of carefully curated human motion data for training the humanoid teleoperation policy. We design a transformer-based policy and train it for over 1300 GPU hours. The policy is deployed on a full-sized humanoid with 31 DoF (excluding hands). Both simulation and real-world experiments verify high-dynamic motion, high-precision tracking, and strong robustness in sim-to-real humanoid teleoperation. Motion data, demos and code can be found in our website.

Problem

• humanoid teleoperation
• global pose drift
• whole-body motion tracking
• long-horizon control
• sim-to-real transfer

Innovation

• closed-loop global tracking
• humanoid teleoperation
• drift-free motion mimicry
• data-driven randomization
• adversarial motion prior