Achieving Human Level Competitive Robot Table Tennis

2024-08-07

arXiv.org

AI Summary

This work addresses the challenge of achieving human-level real-time performance for robots in realistic adversarial tasks, focusing on table tennis—a highly dynamic, interactive domain. Methodologically, we propose the first end-to-end learning-based robotic system capable of sustaining stable amateur-human competitive proficiency. Our approach features a hierarchical reinforcement learning architecture comprising a physics-aware low-level skill controller and a high-level skill selector; introduces the first automatic curriculum learning framework grounded in real-task distribution, coupled with zero-shot sim-to-real transfer; and incorporates online opponent modeling and real-time skill switching to enable closed-loop visuomotor control. In 29 human-robot matches, the system achieved a 45% win rate overall (100% against beginners, 55% against intermediate players), demonstrating breakthrough capabilities in adapting to unknown opponent behaviors, dynamically scaling task difficulty, and seamless simulation-to-reality transfer.

Technology Category

• Multiagent Systems: Adversarial Agents
• Intelligent Robots: Manipulation
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

Achieving human-level speed and performance on real world tasks is a north star for the robotics research community. This work takes a step towards that goal and presents the first learned robot agent that reaches amateur human-level performance in competitive table tennis. Table tennis is a physically demanding sport which requires human players to undergo years of training to achieve an advanced level of proficiency. In this paper, we contribute (1) a hierarchical and modular policy architecture consisting of (i) low level controllers with their detailed skill descriptors which model the agent's capabilities and help to bridge the sim-to-real gap and (ii) a high level controller that chooses the low level skills, (2) techniques for enabling zero-shot sim-to-real including an iterative approach to defining the task distribution that is grounded in the real-world and defines an automatic curriculum, and (3) real time adaptation to unseen opponents. Policy performance was assessed through 29 robot vs. human matches of which the robot won 45% (13/29). All humans were unseen players and their skill level varied from beginner to tournament level. Whilst the robot lost all matches vs. the most advanced players it won 100% matches vs. beginners and 55% matches vs. intermediate players, demonstrating solidly amateur human-level performance. Videos of the matches can be viewed at https://sites.google.com/view/competitive-robot-table-tennis

Problem

• Achieving human-level robot performance in competitive table tennis
• Bridging sim-to-real gap with hierarchical policy architecture
• Enabling real-time adaptation to unseen opponents

Innovation

• Hierarchical modular policy architecture for robot control
• Zero-shot sim-to-real with automatic task curriculum
• Real-time adaptation to unseen opponents