HEFT: Heavy-Payload Full-size Humanoid Teleoperation with Privileged Motion Guidance and Windowed Payload Curriculum

2026-07-02

AI Summary

This work addresses the challenges of motion tracking noise and underutilized payload capacity in full-scale humanoid robots during heavy-load teleoperation. To overcome these limitations, the authors propose Privileged Motion Guidance (PMG) and a Weighted Payload Curriculum (WPC): PMG learns physically plausible reference motions from noisy VR input, while WPC enhances robustness through expert-guided, progressive payload training. The approach achieves, for the first time, stable teleoperation on the 175-cm, 65-kg L7 humanoid robot under payloads as high as 24 kg, successfully executing complex maneuvers such as turning, walking, and deep squats. This significantly improves motion tracking fidelity and system stability in heavy-load scenarios.

Abstract

General motion tracking and teleoperation offer a promising path to scalable humanoid skill acquisition, yet most existing frameworks are validated on compact platforms or without real payload interaction, leaving full-size humanoids with real payloads largely unexplored. Scaling to full-size humanoids introduces two compounding challenges: their larger inertia and tighter balance margins make tracking highly sensitive to noise, drift, and retargeting errors from commodity VR trackers, while their payload potential remains largely underutilized. We present HEFT, a heavy-payload full-size humanoid teleoperation framework that addresses both challenges. HEFT learns from deployable noisy VR references with physically plausible reconstructed references through Privileged Motion Guidance (PMG), and uses a Windowed Payload Curriculum (WPC) with expert-guided payload caps to acquire robust heavy-payload tracking. We deploy HEFT on L7, a 175cm, 65kg humanoid. The robot tracks motions including turns, forward/backward locomotion, and squats under payloads up to 24kg.

Problem

• humanoid teleoperation
• heavy-payload
• motion tracking
• full-size humanoid
• VR-based control

Innovation

• Privileged Motion Guidance
• Windowed Payload Curriculum
• Heavy-Payload Teleoperation
• Full-size Humanoid
• VR-based Motion Tracking