Weave: Learning Whole-Body Dexterous Loco-Manipulation from Human-Object Interactions

2026-09-15

AI Summary

本文提出Weave框架，通过学习人类演示来解决人形机器人与物体交互中的全身协调控制问题，实现了高成功率的复杂互动。

Abstract

Learning humanoid-object interaction requires coordinating whole-body balance, locomotion, and dexterous hand contact to control both robot and object motion. Human demonstrations provide examples of coordinated interaction, but transferring these behaviors to humanoid robots requires learning how to establish and maintain effective contacts under different embodiments and dynamics. We present Weave, a unified framework for learning whole-body dexterous humanoid-object interaction from captured human demonstrations. Weave first converts captured human-object interactions into executable robot-object references through contact-aware retargeting and approach-motion completion. At its core is a contact- and geometry-aware policy that jointly commands 29 body joints and 12 actuated finger joints across multiple objects and interaction sequences. Evaluation across nine objects yields a 92.5% success rate on trained interactions and, without any additional training, 65.0% on sequences never seen during training. We additionally release ~9,000 physically executed rollouts spanning ~23 hours, providing robot-object trajectories with contact annotations for downstream interaction-policy learning and physically consistent HOI motion generation. Project website: https://xiaohu-art.github.io/Weave/

Problem

• humanoid-object interaction
• whole-body dexterous manipulation
• demonstration learning

Innovation

• whole-body dexterous interaction
• contact-aware retargeting
• geometry-aware policy