🤖 AI Summary
Existing methods predominantly focus on single tasks—such as trajectory prediction, imputation, or spatiotemporal recovery—and rely on pedestrian-centric benchmarks featuring largely random motion, which poorly reflect the highly interactive, structured multi-agent dynamics prevalent in sports. To address this gap, we propose the first unified multi-task framework tailored specifically for sports analytics, jointly tackling trajectory prediction, imputation, and spatiotemporal recovery. Our key contributions include: (i) the Ghost Spatial Masking (GSM) module to capture dynamic spatial dependencies; (ii) Bidirectional Temporal Mamba (BTM) and Bidirectional Temporal Scaling (BTS) mechanisms to enhance long-range temporal modeling; and (iii) three new large-scale sports benchmarks—Basketball-U, Football-U, and Soccer-U. Extensive experiments demonstrate consistent and significant improvements over state-of-the-art methods across all datasets, particularly in long-horizon interaction modeling and missing trajectory reconstruction. Code, models, and datasets are publicly released.
📝 Abstract
Understanding multi-agent movement is critical across various fields. The conventional approaches typically focus on separate tasks such as trajectory prediction, imputation, or spatial-temporal recovery. Considering the unique formulation and constraint of each task, most existing methods are tailored for only one, limiting the ability to handle multiple tasks simultaneously, which is a common requirement in real-world scenarios. Another limitation is that widely used public datasets mainly focus on pedestrian movements with casual, loosely connected patterns, where interactions between individuals are not always present, especially at a long distance, making them less representative of more structured environments. To overcome these limitations, we propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs, adaptable to diverse scenarios in the domain of sports games. Specifically, we introduce a Ghost Spatial Masking (GSM) module, embedded within a Transformer encoder, for spatial feature extraction. We further extend recent State Space Models (SSMs), known as the Mamba model, into a Bidirectional Temporal Mamba (BTM) to better capture temporal dependencies. Additionally, we incorporate a Bidirectional Temporal Scaled (BTS) module to thoroughly scan trajectories while preserving temporal missing relationships. Furthermore, we curate and benchmark three practical sports datasets, Basketball-U, Football-U, and Soccer-U, for evaluation. Extensive experiments demonstrate the superior performance of our model. We hope that our work can advance the understanding of human movement in real-world applications, particularly in sports. Our datasets, code, and model weights are available here https://github.com/colorfulfuture/UniTraj-pytorch.