Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation

2025-11-25

AI Summary

Existing methods rely on synthetic data and struggle to accurately estimate 3D trajectories and spin of table tennis balls in real monocular videos, primarily due to the absence of ground-truth 3D trajectory and spin annotations in realistic scenes. This work proposes a two-stage framework: a front-end that robustly detects ball instances and table keypoints to yield reliable 2D observations; and a back-end physics-guided 3D lifting network trained jointly on high-quality real 2D annotations and physically synthesized 3D data, effectively decoupling perception from reconstruction. The design significantly improves robustness against detection failures, variable frame rates, and localization errors in ball or table detection. Evaluated on real-world videos, our method achieves millimeter-level 3D trajectory recovery and high-precision spin estimation, outperforming state-of-the-art approaches and demonstrating practical deployability.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Obtaining the precise 3D motion of a table tennis ball from standard monocular videos is a challenging problem, as existing methods trained on synthetic data struggle to generalize to the noisy, imperfect ball and table detections of the real world. This is primarily due to the inherent lack of 3D ground truth trajectories and spin annotations for real-world video. To overcome this, we propose a novel two-stage pipeline that divides the problem into a front-end perception task and a back-end 2D-to-3D uplifting task. This separation allows us to train the front-end components with abundant 2D supervision from our newly created TTHQ dataset, while the back-end uplifting network is trained exclusively on physically-correct synthetic data. We specifically re-engineer the uplifting model to be robust to common real-world artifacts, such as missing detections and varying frame rates. By integrating a ball detector and a table keypoint detector, our approach transforms a proof-of-concept uplifting method into a practical, robust, and high-performing end-to-end application for 3D table tennis trajectory and spin analysis.

Problem

• Estimating precise 3D ball trajectory and spin from monocular videos
• Overcoming poor generalization from synthetic to real-world data
• Handling real-world artifacts like missing detections and frame rates

Innovation

• Two-stage pipeline divides front-end perception and back-end uplifting
• Front-end uses 2D supervision from TTHQ dataset
• Back-end trained on physically-correct synthetic data