🤖 AI Summary
This work addresses the challenge of modeling heterogeneous features in soccer event data, where traditional one-hot or ordinal encodings fail to capture semantic relationships among categorical variables. The study proposes the first application of TabTransformer to soccer event representation learning, leveraging learnable dense embeddings and self-attention mechanisms to capture sport-specific action semantics during pretraining and generate general-purpose event representations. By effectively modeling latent dependencies among categorical features, the approach significantly outperforms baseline models on downstream tasks such as action value estimation and playing style recognition. Moreover, improved Brier scores demonstrate that the model yields better-calibrated predictive probabilities.
📝 Abstract
Football event data constitute a rich spatiotemporal source for quantitative analysis of player actions in team sports. These datasets contain heterogeneous features, combining continuous location coordinates with categorical variables such as action type, action outcome, and body part. Such data have been applied in sports analytics for match outcome forecasting, player evaluation, and tactical pattern recognition. However, existing approaches predominantly encode categorical features using one-hot or ordinal embedding representations, overlooking the intrinsic semantics of action descriptors. The Transformer is a deep neural network architecture based on self-attention that captures dependencies between input features at arbitrary positions. We propose and implement a Transformer-based model to learn latent dependencies among categorical event features and produce dense representations of football events. By encoding categorical features as learned embedding vectors, sport-specific action semantics are captured during pretraining, enabling the representations to support downstream tasks such as action value estimation and play style recognition. Empirical evaluation shows that the embedding representations yield superior probability calibration over task-specific baselines on the downstream prediction tasks, as measured by Brier score.