VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids

2026-05-02

AI Summary

This work addresses the challenge of robustly pushing large objects with unknown mass and friction to arbitrary target poses using only onboard vision on humanoid robots. To this end, the authors propose a two-layer hierarchical system: a high-level policy generates goal-conditioned motion commands from noisy egocentric visual observations, while a low-level force-adaptive whole-body controller dynamically compensates for variations in object physical properties, enabling closed-loop, goal-directed pushing. This approach represents the first integration of onboard egocentric vision with force-adaptive control, operating without access to privileged state information and effectively handling objects with unknown dynamics. In experiments, the method achieves over 90% success in simulation and exceeds 80% success on a real robot, successfully manipulating objects weighing up to 17 kg—more than half the robot’s own mass.

Abstract

The ability to push large objects in a goal-directed manner using onboard egocentric perception is an essential skill for humanoid robots to perform complex tasks such as material handling in warehouses. To robustly manipulate heavy objects to arbitrary goal configurations, the robot must cope with unknown object mass and ground friction, noisy onboard perception, and actuation errors; all in a real-time feedback loop. Existing solutions either rely on privileged object-state information without onboard perception or lack robustness to variations in goal configurations and object physical properties. In this work, we present VOFA, a visual goal-conditioned humanoid loco-manipulation system capable of pushing objects with unknown physical properties to arbitrary goal positions. VOFA consists of a two-level hierarchical architecture with a high-level visuomotor policy and a low-level force-adaptive whole-body controller. The high-level policy processes noisy onboard observations and generates goal-conditioned commands to operate in closed loop across diverse object-goal configurations, while the low-level whole-body controller provides robustness to variations in object physical properties. VOFA is extensively evaluated in both simulation and real-world experiments on the Booster T1 humanoid robot. Our results demonstrate strong performance, achieving over 90% success in simulation and over 80% success in real-world trials. Moreover, VOFA successfully pushes objects weighing up to 17kg, exceeding half of the Booster T1's body weight.

Problem

• humanoid robots
• visual object pushing
• goal-directed manipulation
• onboard perception
• unknown physical properties

Innovation

• visual goal-conditioned control
• force-adaptive whole-body control
• humanoid loco-manipulation
• onboard egocentric perception
• unknown object properties