Event-based Gaze Control System for Accurate Real-time Spin Estimation in Professional Ball Games

2026-06-25

AI Summary

This work addresses the challenge of accurately estimating the spin of fast-moving, small, and rapidly rotating balls in real time—a task beyond the capabilities of conventional cameras. The authors present the first event-camera-based active vision system, integrating a high-speed pan-tilt mirror with a tunable-focus telephoto lens to enable real-time tracking and spin estimation of unmarked balls. Their approach employs spherical contrast maximization (s-CMax) during an offline phase for high-precision spin estimation, while the online phase combines an uncertainty-aware convolutional neural network with GPU-accelerated optimization to achieve low-latency inference. Evaluated across multiple ball types, the system attains offline errors of 2.1% in magnitude and 4.0° in axis orientation. In live professional table tennis matches, it achieves real-time spin estimation accuracy of 8.8% and 6.4° with only 3 ms latency and a throughput of 750 Hz.

Abstract

Spin plays a crucial role in many ball sports due to its effect on the trajectory of the ball. Vision-based estimation of the ball's spin during a game with conventional cameras is challenging due to the ball's small size, high speed, and fast rotation. To address these challenges, we propose an event-based active vision system that can track unmodified balls and measure their spin in real-time. The system consists of an event camera for its high temporal resolution and minimal motion blur, high-speed pan/tilt galvanometer mirrors to keep the ball in the field of view, and a low-latency focus-tunable telephoto lens to increase the spatial resolution on the ball and keep it in focus. To track the ball, we use a hybrid approach that combines 2D event-based detection for centering and 3D positions from a ball localization system for re-initialization. For high-accuracy spin estimation, we propose an offline method that performs contrast maximization on the sphere (s-CMax). This method achieves state-of-the-art accuracy on static balls across multiple sports (table tennis, baseball, tennis, and golf), with mean magnitude and axis errors of 2.1% and 4.0 degrees, respectively. We then develop a low-latency online method for table tennis as a case study in real-time applications. This method uses an uncertainty-aware convolutional neural network trained on pseudo-ground-truth spin labels from the offline approach, combined with a GPU-accelerated batch implementation of contrast maximization for refinement. We demonstrate reliable tracking and spin estimation with a three-view setup during professional table tennis matches, with high accuracy (8.8% magnitude and 6.4 degrees axis mismatch), 3 ms latency, and 750 Hz throughput.

Problem

• spin estimation
• event-based vision
• real-time tracking
• ball sports
• high-speed motion

Innovation

• event-based vision
• spin estimation
• active gaze control
• contrast maximization
• real-time tracking