LATTE-MV: Learning to Anticipate Table Tennis Hits from Monocular Videos

2025-03-26

AI Summary

Real-time intent prediction in high-speed adversarial table tennis is challenging due to stringent latency constraints and limited training data, leading to poor generalization. Method: This paper proposes a forward-looking framework for shot-intent prediction and control from monocular video. It integrates scalable monocular 3D match reconstruction, motion trajectory forecasting, uncertainty-aware Bayesian neural networks, and a reinforcement learning controller to enable end-to-end inference from visual input to shot-intent execution. Contribution/Results: The core innovation is the first uncertainty modeling mechanism enabling robust intent prediction under few-shot conditions, coupled with a synergistic forward-looking control policy. In simulation, the framework increases rally success rate from 49.9% to 59.0%, significantly outperforming non-foresighted baselines—demonstrating substantial improvements in both reaction timeliness and decision reliability.

Technology Category

• Multiagent Systems: Adversarial Agents
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Physical agility is a necessary skill in competitive table tennis, but by no means sufficient. Champions excel in this fast-paced and highly dynamic environment by anticipating their opponent's intent - buying themselves the necessary time to react. In this work, we take one step towards designing such an anticipatory agent. Previous works have developed systems capable of real-time table tennis gameplay, though they often do not leverage anticipation. Among the works that forecast opponent actions, their approaches are limited by dataset size and variety. Our paper contributes (1) a scalable system for reconstructing monocular video of table tennis matches in 3D and (2) an uncertainty-aware controller that anticipates opponent actions. We demonstrate in simulation that our policy improves the ball return rate against high-speed hits from 49.9% to 59.0% as compared to a baseline non-anticipatory policy.

Problem

• Anticipating opponent intent in table tennis
• Reconstructing 3D from monocular match videos
• Improving ball return rate with anticipation

Innovation

• 3D reconstruction from monocular videos
• Uncertainty-aware action anticipation
• Improved ball return rate