CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulation

2026-06-22

AI Summary

This work addresses the challenge of performing high-degree-of-freedom dexterous manipulation while a humanoid robot is in motion, moving beyond the conventional “walk–stop–manipulate” paradigm. The authors propose a latent-space residual control framework grounded in coordinated body–hand priors, featuring a shared task-context latent interface and decoupled residual heads to enable reliable synergy between whole-body locomotion and a 20-DoF dexterous hand during walking. By integrating a privileged teacher network, proprioception-conditioned latent prior distillation, and residual reinforcement learning atop a frozen prior, the approach substantially enhances trainability for contact-intensive tasks. Validated on the Unitree G1 platform, the method successfully executes complex loco-manipulation tasks such as grasping a bottle and opening a refrigerator door while walking. Ablation studies demonstrate its clear superiority over baselines like joint-space PPO under identical reward budgets.

Abstract

Humanoid loco-manipulation is often simplified into a stop-and-go process: walking to an object, stopping to manipulate it, and then resuming locomotion. It also commonly relies on low degree-of-freedom (DoF) end effectors that behave like an open-close grasp primitive. We introduce CoorDex, a learning pipeline that converts high-dimensional body and dexterous hand control into coordinated latent residual control, enabling high-DoF dexterous loco-manipulation on the move. Starting from simulated whole-body and hand demonstrations, CoorDex trains privileged motion tracking teachers for the humanoid body and dexterous hand, distills them into proprioception-conditioned latent priors, and uses the frozen priors as the action space for downstream residual reinforcement learning. A coordinated latent residual policy composes these priors through shared task context and separate body-hand residual heads, preserving natural whole-body motion while improving finger-level contact reliability. CoorDex enables a Unitree G1 humanoid with a 20-DoF WUJI hand to execute dexterous manipulation while in motion, including non-stop bottle grasping and carrying, fridge door opening on the move, and cube pick-and-turn. Ablations on the walk-grasp-carry task show that joint-space PPO, joint-space hand control, and monolithic latent prediction all fail under the same reward budget, while the latent-prior interface and coordinated residual structure make high-dimensional contact-rich loco-manipulation trainable. Project Page: https://skevinci.github.io/coordex/

Problem

• humanoid loco-manipulation
• dexterous manipulation
• high-DoF hand
• continuous motion
• body-hand coordination

Innovation

• dexterous loco-manipulation
• latent residual control
• coordinated priors
• whole-body control
• proprioception-conditioned policy