Embrace Collisions: Humanoid Shadowing for Deployable Contact-Agnostics Motions

2025-02-03

AI Summary

Enabling safe, robust whole-body locomotion for humanoid robots under unknown physical contacts remains a critical challenge. Method: We propose the first general-purpose motion framework designed for real-world interaction, integrating GPU-accelerated rigid-body simulation, end-to-end whole-body proprioceptive reinforcement learning, disturbance-resilient real-time motion tracking, and a lightweight collision-aware state representation. Contribution/Results: Our approach achieves contact-agnostic whole-body proprioceptive control—the first of its kind—overcoming key bottlenecks in zero-shot sim-to-real transfer induced by arbitrary contact sequences and extreme base rotations (>180°). It supports discrete high-level command execution and adaptive rejection of infeasible commands. Deployed on a physical humanoid robot without fine-tuning, it successfully executes diverse contact-rich whole-body behaviors—including sit-to-stand, rolling, and ground crawling—demonstrating unprecedented generalization and stability in complex, dynamic environments.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Motion & Tracking

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Previous humanoid robot research works treat the robot as a bipedal mobile manipulation platform, where only the feet and hands contact the environment. However, we humans use all body parts to interact with the world, e.g., we sit in chairs, get up from the ground, or roll on the floor. Contacting the environment using body parts other than feet and hands brings significant challenges in both model-predictive control and reinforcement learning-based methods. An unpredictable contact sequence makes it almost impossible for model-predictive control to plan ahead in real time. The success of the zero-shot sim-to-real reinforcement learning method for humanoids heavily depends on the acceleration of GPU-based rigid-body physical simulator and simplification of the collision detection. Lacking extreme torso movement of the humanoid research makes all other components non-trivial to design, such as termination conditions, motion commands and reward designs. To address these potential challenges, we propose a general humanoid motion framework that takes discrete motion commands and controls the robot's motor action in real time. Using a GPU-accelerated rigid-body simulator, we train a humanoid whole-body control policy that follows the high-level motion command in the real world in real time, even with stochastic contacts and extremely large robot base rotation and not-so-feasible motion command. More details at https://project-instinct.github.io

Problem

• Robotics
• Tactile Sensing
• Complex Motion Control

Innovation

• Whole-body Collision Processing
• Real-time Command Execution
• Adaptive Human-like Motion