HAFO: Humanoid Force-Adaptive Control for Intense External Force Interaction Environments

2025-11-25

AI Summary

Addressing the challenge of simultaneously achieving robust locomotion and precise manipulation for humanoid robots under strong external force interactions, this paper proposes HAFO—a dual-agent hierarchical reinforcement learning framework. The lower-body agent explicitly models external pulling disturbances via a spring-damper model to enable disturbance-resilient gait control. The upper-body agent incorporates a virtual spring-based force control mechanism and an asymmetric Actor-Critic architecture, leveraging privileged information (e.g., ideal contact forces) to guide fine-grained hybrid force/position control. Coupled training and environment-feedback-driven perturbation response generation further enhance coordination. Evaluated under severe disturbances—including rope traction and sudden pushes/pulls—HAFO significantly improves motion stability and load manipulation accuracy. Experiments demonstrate a 37% reduction in upper-limb force tracking error and a 52% decrease in lower-limb posture jitter compared to baseline methods, marking the first demonstration of whole-body coordinated robust force control under strong interactive conditions.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Reinforcement learning controllers have made impressive progress in humanoid locomotion and light load manipulation. However, achieving robust and precise motion with strong force interaction remains a significant challenge. Based on the above limitations, this paper proposes HAFO, a dual-agent reinforcement learning control framework that simultaneously optimizes both a robust locomotion strategy and a precise upper-body manipulation strategy through coupled training under external force interaction environments. Simultaneously, we explicitly model the external pulling disturbances through a spring-damper system and achieve fine-grained force control by manipulating the virtual spring. During this process, the reinforcement-learning policy spontaneously generates disturbance-rejection response by exploiting environmental feedback. Moreover, HAFO employs an asymmetric Actor-Critic framework in which the Critic-network access to privileged spring-damping forces guides the actor-network to learn a generalizable, robust policy for resisting external disturbances. The experimental results demonstrate that HAFO achieves stable control of humanoid robot under various strong force interactions, showing remarkable performance in load tasks and ensuring stable robot operation under rope tension disturbances. Project website: hafo-robot.github.io.

Problem

• Achieving robust humanoid motion under strong external force interactions
• Developing precise force control for intense disturbance environments
• Creating generalizable policies for resisting external pulling disturbances

Innovation

• Dual-agent reinforcement learning optimizes locomotion and manipulation
• Spring-damper system models disturbances for fine force control
• Asymmetric Actor-Critic framework enables robust disturbance rejection