1000 Rallies: An Event-Camera Dataset and Real-Time Learned Ball-State Estimation for Robotic Table Tennis

2026-06-24

AI Summary

This work addresses the limitations of conventional frame-based cameras in high-speed table tennis trajectory estimation—namely, temporal blind spots and high computational costs—and the absence of large-scale event camera datasets tailored to real-world sports scenarios. To bridge this gap, the authors introduce the first large-scale table tennis event camera dataset, comprising over 1,000 rally segments accompanied by synchronized 1 kHz high-speed video. They further propose a real-time perception framework that integrates convolutional neural networks with Kalman filtering to jointly estimate the ball’s position and velocity directly from event streams. Compared to baseline methods relying solely on positional information, the proposed approach reduces bounce-point prediction error by 36% and enables, for the first time, a closed-loop human–robot table tennis system driven entirely by event camera input.

Abstract

Robotic table tennis has emerged as a compelling benchmark for real-time robotic perception due to its fast ball dynamics and stringent timing requirements. Accurate, high-frequency, and low-latency ball state estimation is critical for reliable trajectory prediction and timely control. Traditional frame-based cameras face an inherent trade-off: low frame rates leave temporal blind spots that miss fast-moving objects and high frame rates raise data and computational cost. Event cameras instead offer microsecond temporal resolution and, under sufficient illumination, remain largely free of motion blur even at high ball speeds. However, the community lacks large-scale datasets to develop and benchmark event-based perception in realistic sports scenarios. We address this gap by introducing the first large-scale event-camera dataset for table tennis, comprising over 1000 rallies from a diverse group of players ranging from amateurs to elite-level athletes. Each recording captures the event stream alongside 14 synchronized high-speed frame-based cameras at 200 FPS, which we use to produce 1 kHz pseudo ground-truth labels for ball position, velocity, and spin. Building on this dataset, we train a convolutional neural network robust to background player motion that jointly estimates the ball's position and velocity in the image-plane from events. Treating the predicted velocity as an additional measurement in the Kalman filter reduces bounce-point prediction error by 36% relative to a position-only baseline. Finally, we close the perception-action loop by integrating the event-based system with a Stäubli robotic arm, enabling the first real-time human-robot table tennis rallies driven by event-based perception.

Problem

• event camera
• ball-state estimation
• robotic table tennis
• dataset
• real-time perception

Innovation

• event camera
• ball-state estimation
• table tennis robotics
• high-speed perception
• Kalman filtering