CWI: Composite Humanoid Whole-Body Imitation System for Loco-manipulation

2026-06-25

AI Summary

This work addresses the challenge of achieving both stable locomotion and dexterous manipulation in humanoid robots under sparse rewards, a task further complicated by motion capture data imbalance that often leads to uncoordinated behaviors in existing imitation learning approaches. To overcome these limitations, the authors propose CWI, a composite whole-body imitation framework that decouples upper-body manipulation from lower-body locomotion. The upper body imitates diverse motion capture data, while the lower body leverages a dual-discriminator architecture guided by adversarial motion priors (AMP) to ensure stable command-following. Multi-critic reinforcement learning mitigates conflicting objectives, and a teacher–student distillation scheme yields a lightweight policy dependent only on hand poses and velocity/height commands. Experiments on the LimX Oli robot demonstrate robust loco-manipulation capabilities, superior whole-body coordination, and practical teleoperation performance without requiring full-body motion capture.

Abstract

Achieving everyday tasks with humanoid robots requires coordinating stable locomotion with versatile manipulation. However, existing whole-body controllers still face significant challenges. Methods trained solely via command sampling, without motion-capture (MoCap) data, often struggle with sparse rewards and require carefully tuned curricula to converge. This is especially problematic for upper-body control, where the resulting motions deviate from human-like statistics and degrade whole-body coordination. Conversely, approaches that imitate full-body MoCap data suffer from dataset imbalance, as many locomotion trajectories are overly aggressive for stable-locomotion scenarios, necessitating extensive data filtering and augmentation. To address this, we present Composite Whole-Body Imitation (CWI), a framework that decouples the use of MoCap data for upper-body manipulation and lower-body locomotion. This decoupling allows us to exploit the full MoCap dataset of diverse manipulation references, while stable, command-conditioned lower-body locomotion is guided by dual discriminators trained on curated expert-quality walking and squatting clips via an Adversarial Motion Prior (AMP). A multi-critic architecture reduces conflicts among locomotion, manipulation, and motion-style objectives, and a teacher--student distillation stage yields a whole-body policy conditioned only on bimanual hand poses and velocity/height commands. We evaluate CWI through simulation experiments and real-world deployment on a full-size LimX Oli humanoid. The results show competitive loco-manipulation performance, robust whole-body coordination, and practical teleoperation without full-body motion-capture equipment. A project page with supplementary material can be found at https://cwi-ral.github.io/CWI-RAL-Webpage.

Problem

• humanoid robots
• whole-body control
• motion capture
• loco-manipulation
• imitation learning

Innovation

• whole-body imitation
• motion capture decoupling
• adversarial motion prior
• multi-critic architecture
• teacher-student distillation