HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning

2025-08-28

AI Summary

Humanoid robots face significant challenges in high-speed dynamic manipulation tasks—such as table tennis—requiring sub-second perception-decision-action closed-loop latency. To address this, we propose a hierarchical real-time interaction framework: (i) a low-level layer integrating model-based trajectory planning with reinforcement learning–driven whole-body motion control; (ii) a mid-level layer incorporating human motion priors to generate naturalistic striking behaviors; and (iii) a high-level layer unifying visual perception and ball trajectory prediction for closed-loop response. Our approach achieves, for the first time on a general-purpose humanoid platform, continuous human-robot rally of 106 strokes and autonomous dual-robot competition. These results demonstrate exceptional agility, robustness, and natural interactive capability in real-world settings. The framework establishes a scalable technical pathway for real-time embodied intelligence in dynamic environments.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

Humanoid robots have recently achieved impressive progress in locomotion and whole-body control, yet they remain constrained in tasks that demand rapid interaction with dynamic environments through manipulation. Table tennis exemplifies such a challenge: with ball speeds exceeding 5 m/s, players must perceive, predict, and act within sub-second reaction times, requiring both agility and precision. To address this, we present a hierarchical framework for humanoid table tennis that integrates a model-based planner for ball trajectory prediction and racket target planning with a reinforcement learning-based whole-body controller. The planner determines striking position, velocity and timing, while the controller generates coordinated arm and leg motions that mimic human strikes and maintain stability and agility across consecutive rallies. Moreover, to encourage natural movements, human motion references are incorporated during training. We validate our system on a general-purpose humanoid robot, achieving up to 106 consecutive shots with a human opponent and sustained exchanges against another humanoid. These results demonstrate real-world humanoid table tennis with sub-second reactive control, marking a step toward agile and interactive humanoid behaviors.

Problem

• Develops a humanoid robot for table tennis with fast reactions
• Integrates hierarchical planning and learning for dynamic control
• Achieves real-time performance in human-robot table tennis rallies

Innovation

• Hierarchical planning and learning framework
• Model-based planner for trajectory prediction
• Reinforcement learning-based whole-body controller