TeleGate: Whole-Body Humanoid Teleoperation via Gated Expert Selection with Motion Prior

2026-02-10

AI Summary

This work addresses the performance degradation of existing unified controllers in high-dynamic teleoperation of human motions, where balancing accuracy and generalization remains challenging. The authors propose a lightweight unified framework that employs a gating network to dynamically select among domain-specific expert policies in real time. To enable anticipatory control without requiring future trajectory inputs, the framework incorporates a VAE-based motion prior module that predicts future action intent. This design preserves the full capabilities of individual expert policies, circumventing the performance loss typically incurred by knowledge distillation. Trained on only 2.5 hours of motion capture data, the method achieves high-fidelity, real-time reproduction of complex dynamic behaviors—including running, jumping, and fall recovery—on both simulation and the Unitree G1 robot, significantly outperforming current baselines.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Real-time whole-body teleoperation is a critical method for humanoid robots to perform complex tasks in unstructured environments. However, developing a unified controller that robustly supports diverse human motions remains a significant challenge. Existing methods typically distill multiple expert policies into a single general policy, which often inevitably leads to performance degradation, particularly on highly dynamic motions. This paper presents TeleGate, a unified whole-body teleoperation framework for humanoid robots that achieves high-precision tracking across various motions while avoiding the performance loss inherent in knowledge distillation. Our key idea is to preserve the full capability of domain-specific expert policies by training a lightweight gating network, which dynamically activates experts in real-time based on proprioceptive states and reference trajectories. Furthermore, to compensate for the absence of future reference trajectories in real-time teleoperation, we introduce a VAE-based motion prior module that extracts implicit future motion intent from historical observations, enabling anticipatory control for motions requiring prediction such as jumping and standing up. We conducted empirical evaluations in simulation and also deployed our technique on the Unitree G1 humanoid robot. Using only 2.5 hours of motion capture data for training, our TeleGate achieves high-precision real-time teleoperation across diverse dynamic motions (e.g., running, fall recovery, and jumping), significantly outperforming the baseline methods in both tracking accuracy and success rate.

Problem

• whole-body teleoperation
• humanoid robots
• expert policies
• motion prior
• real-time control

Innovation

• gated expert selection
• motion prior
• whole-body teleoperation
• humanoid robot
• anticipatory control