Trajectory Imputation in Multi-Agent Sports with Derivative-Accumulating Self-Ensemble

2024-08-20

AI Summary

To address the problem of physically implausible trajectory predictions in team sports—caused by frequent missing observations and complex dynamic multi-agent interactions—this paper proposes MIDAS, a novel framework based on the Set Transformer for joint prediction of position, velocity, and acceleration. MIDAS introduces a derivative-recursive cumulative self-ensembling mechanism that leverages physical kinematic constraints across multiple orders to generate diverse yet physically consistent trajectory estimates, which are then fused via learnable weighted aggregation. This is the first approach to achieve coordinated optimization of kinematic variables and enforce trajectory-level physical consistency in multi-agent sports settings. Evaluated on three benchmark datasets, MIDAS reduces positional prediction error by up to 32% and improves acceleration consistency by 41%, leading to significant gains in downstream tasks such as total distance covered estimation and pass success prediction.

Technology Category

• Multiagent Systems: Teamwork
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Motion & Tracking

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multi-agent trajectory data collected from domains such as team sports often suffer from missing values due to various factors. While many imputation methods have been proposed for spatiotemporal data, they are not well-suited for multi-agent sports scenarios where player movements are highly dynamic and inter-agent interactions continuously evolve. To address these challenges, we propose MIDAS (Multi-agent Imputer with Derivative-Accumulating Self-ensemble), a framework that imputes multi-agent trajectories with high accuracy and physical plausibility. It jointly predicts positions, velocities, and accelerations through a Set Transformer-based neural network and generates alternative estimates by recursively accumulating predicted velocity and acceleration values. These predictions are then combined using a learnable weighted ensemble to produce final imputed trajectories. Experiments on three sports datasets demonstrate that MIDAS significantly outperforms existing baselines in both positional accuracy and physical plausibility. Lastly, we showcase use cases of MIDAS, such as approximating total distance and pass success probability, to highlight its applicability to practical downstream tasks that require complete tracking data.

Problem

• Imputing missing values in multi-agent sports trajectory data
• Handling highly dynamic player movements and inter-agent interactions
• Ensuring high accuracy and physical plausibility in imputed trajectories

Innovation

• Set Transformer-based neural network for joint predictions
• Recursive derivative accumulation for alternative estimates
• Learnable weighted ensemble for final imputation