FALCON: Learning Force-Adaptive Humanoid Loco-Manipulation

2025-05-10

AI Summary

Humanoid robots still face significant challenges in whole-body motion-manipulation coordination under 3D end-effector force interaction, including low tracking accuracy, poor robustness against disturbances, and limited generalization across tasks and platforms. To address these issues, this work proposes a dual-agent decoupled architecture coupled with a force-aware adaptive training mechanism. The method integrates a reinforcement learning-based two-agent framework, force-progressive curriculum learning, implicit force-compensation control, and Sim2Real transfer—enabling cross-robot deployment without platform-specific reward engineering. Our approach achieves up to twofold improvement in upper-limb joint tracking accuracy, ensures disturbance-resilient bipedal locomotion, and accelerates policy convergence. Experimental validation on a physical humanoid robot demonstrates successful execution of dynamic force-intensive tasks—such as object transport, cart pulling, and door opening—across a 0–100 N force range.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Humanoid loco-manipulation holds transformative potential for daily service and industrial tasks, yet achieving precise, robust whole-body control with 3D end-effector force interaction remains a major challenge. Prior approaches are often limited to lightweight tasks or quadrupedal/wheeled platforms. To overcome these limitations, we propose FALCON, a dual-agent reinforcement-learning-based framework for robust force-adaptive humanoid loco-manipulation. FALCON decomposes whole-body control into two specialized agents: (1) a lower-body agent ensuring stable locomotion under external force disturbances, and (2) an upper-body agent precisely tracking end-effector positions with implicit adaptive force compensation. These two agents are jointly trained in simulation with a force curriculum that progressively escalates the magnitude of external force exerted on the end effector while respecting torque limits. Experiments demonstrate that, compared to the baselines, FALCON achieves 2x more accurate upper-body joint tracking, while maintaining robust locomotion under force disturbances and achieving faster training convergence. Moreover, FALCON enables policy training without embodiment-specific reward or curriculum tuning. Using the same training setup, we obtain policies that are deployed across multiple humanoids, enabling forceful loco-manipulation tasks such as transporting payloads (0-20N force), cart-pulling (0-100N), and door-opening (0-40N) in the real world.

Problem

• Achieving precise whole-body control with 3D force interaction
• Overcoming limitations in lightweight or quadrupedal platforms
• Enabling robust force-adaptive humanoid loco-manipulation tasks

Innovation

• Dual-agent reinforcement learning framework
• Force curriculum for progressive training
• Implicit adaptive force compensation