HMC: Learning Heterogeneous Meta-Control for Contact-Rich Loco-Manipulation

2025-11-18

AI Summary

In real-world robotic learning from demonstration, pure position control struggles with contact-rich interactions and varying loads. To address this, we propose Heterogeneous Meta-Control (HMC), the first framework enabling continuous, adaptive fusion of position, impedance, and force-position hybrid control modalities directly in torque space. HMC comprises two core components: (i) the HMC-Controller—a multimodal, torque-level hybrid controller integrating heterogeneous control laws via a mixture-of-experts routing mechanism and a unified torque-space action fusion interface; and (ii) the HMC-Policy—a unified, hierarchical policy architecture that jointly leverages large-scale positional trajectories and high-fidelity force-tactile demonstrations through layered policy learning. The framework supports both teleoperation initialization and end-to-end autonomous deployment. Experiments on a humanoid robot demonstrate that HMC improves task success rates by over 50% on contact-intensive tasks—including compliant table wiping and drawer opening—compared to state-of-the-art baselines.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Multimodal Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Learning from real-world robot demonstrations holds promise for interacting with complex real-world environments. However, the complexity and variability of interaction dynamics often cause purely positional controllers to struggle with contacts or varying payloads. To address this, we propose a Heterogeneous Meta-Control (HMC) framework for Loco-Manipulation that adaptively stitches multiple control modalities: position, impedance, and hybrid force-position. We first introduce an interface, HMC-Controller, for blending actions from different control profiles continuously in the torque space. HMC-Controller facilitates both teleoperation and policy deployment. Then, to learn a robust force-aware policy, we propose HMC-Policy to unify different controllers into a heterogeneous architecture. We adopt a mixture-of-experts style routing to learn from large-scale position-only data and fine-grained force-aware demonstrations. Experiments on a real humanoid robot show over 50% relative improvement vs. baselines on challenging tasks such as compliant table wiping and drawer opening, demonstrating the efficacy of HMC.

Problem

• Addressing robot struggles with contacts and varying payloads in complex environments
• Adaptively combining multiple control modalities for loco-manipulation tasks
• Learning robust force-aware policies from position-only and force demonstrations

Innovation

• Adaptively stitches multiple control modalities
• Blends actions from different control profiles
• Unifies controllers with mixture-of-experts routing