Pro-HOI: Perceptive Root-guided Humanoid-Object Interaction

2026-03-01

AI Summary

This work proposes a generalizable loco-manipulation framework for humanoid robots to address the challenges of robust and versatile object interaction under demanding real-world conditions. The approach introduces root trajectory as a unified high-level planning interface, circumventing the need for intricate reward engineering. By leveraging reference motions as intrinsic reward signals, it enables seamless integration of locomotion and manipulation. A continuous object estimation module—fusing real-time perception with a digital twin—is incorporated to support slip detection and autonomous regrasping. Furthermore, Signed Distance Field (SDF)-based loss is employed to mitigate interpenetration during motion execution. Evaluated on the Unitree G1 platform, the system significantly outperforms baseline methods, demonstrating reliable, long-horizon object interaction in complex real-world scenarios.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Executing reliable Humanoid-Object Interaction (HOI) tasks for humanoid robots is hindered by the lack of generalized control interfaces and robust closed-loop perception mechanisms. In this work, we introduce Perceptive Root-guided Humanoid-Object Interaction, Pro-HOI, a generalizable framework for robust humanoid loco-manipulation. First, we collect box-carrying motions that are suitable for real-world deployment and optimize penetration artifacts through a Signed Distance Field loss. Second, we propose a novel training framework that conditions the policy on a desired root-trajectory while utilizing reference motion exclusively as a reward. This design not only eliminates the need for intricate reward tuning but also establishes root trajectory as a universal interface for high-level planners, enabling simultaneous navigation and loco-manipulation. Furthermore, to ensure operational reliability, we incorporate a persistent object estimation module. By fusing real-time detection with Digital Twin, this module allows the robot to autonomously detect slippage and trigger re-grasping maneuvers. Empirical validation on a Unitree G1 robot demonstrates that Pro-HOI significantly outperforms baselines in generalization and robustness, achieving reliable long-horizon execution in complex real-world scenarios.

Problem

• Humanoid-Object Interaction
• loco-manipulation
• closed-loop perception
• generalization
• robustness

Innovation

• Humanoid-Object Interaction
• Root-guided Policy
• Signed Distance Field
• Digital Twin
• Loco-manipulation