Robotic Table Tennis: A Case Study into a High Speed Learning System

2023-07-10

Robotics: Science and Systems

AI Summary

This work addresses the challenges of real-time perception, precise landing-point control, and zero-shot simulation-to-reality transfer in high-speed human–robot table tennis rallies. We propose an end-to-end, low-latency closed-loop learning system. Methodologically, we integrate a lightweight deep vision model with hardware-in-the-loop latency modeling to build a ROS-based real-time motion control architecture; design a hybrid MuJoCo/PandaGym simulation environment featuring reinforcement learning policy optimization and fully automated physical environment reset; and introduce the first zero-shot sim-to-real transfer paradigm enabling direct cross-domain policy deployment. Experiments demonstrate that the physical robot sustains over 120 consecutive rallies, achieves landing-point errors <8 cm, incurs end-to-end latency per cycle <35 ms, and attains >92% zero-shot deployment success rate—marking the first demonstration of centurion-scale, high-precision, fully autonomous human–robot table tennis closed-loop learning.

Abstract

We present a deep-dive into a real-world robotic learning system that, in previous work, was shown to be capable of hundreds of table tennis rallies with a human and has the ability to precisely return the ball to desired targets. This system puts together a highly optimized perception subsystem, a high-speed low-latency robot controller, a simulation paradigm that can prevent damage in the real world and also train policies for zero-shot transfer, and automated real world environment resets that enable autonomous training and evaluation on physical robots. We complement a complete system description, including numerous design decisions that are typically not widely disseminated, with a collection of studies that clarify the importance of mitigating various sources of latency, accounting for training and deployment distribution shifts, robustness of the perception system, sensitivity to policy hyper-parameters, and choice of action space. A video demonstrating the components of the system and details of experimental results can be found at https://youtu.be/uFcnWjB42I0.

Problem

• high-speed robotic learning system
• zero-shot policy transfer training
• latency reduction in robot control

Innovation

• Optimized perception subsystem
• High-speed low-latency controller
• Simulation for zero-shot transfer