PassAI: explainable artificial intelligence algorithm for soccer pass analysis using multimodal information resources

2025-03-11

AI Summary

This work addresses the challenge of limited multimodal interpretability in football pass outcome prediction. We propose PassAI, a novel model that jointly leverages player trajectory images and season-level statistical features—two heterogeneous data modalities—to perform binary classification (pass success/failure) and provide cross-modal attribution explanations. Methodologically, PassAI introduces a dual-stream multimodal interpretable framework comprising a trajectory image encoder, statistical feature embedding module, cross-modal feature alignment mechanism, and SHAP-driven fine-grained attribution analysis. This enables quantitative assessment of relative modality-wise contributions and intra-modal spatial/feature-level visualizations. Evaluated on 6,349 professional match passes, PassAI achieves over a 5% accuracy improvement over state-of-the-art methods. To our knowledge, it is the first football pass analysis system to simultaneously deliver high predictive accuracy and transparent, cross-modal decision reasoning.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

This study developed a new explainable artificial intelligence algorithm called PassAI, which classifies successful or failed passes in a soccer game and explains its rationale using both tracking and passer's seasonal stats information. This study aimed to address two primary challenges faced by artificial intelligence and machine learning algorithms in the sports domain: how to use different modality data for the analysis and how to explain the rationale of the outcome from multimodal perspectives. To address these challenges, PassAI has two processing streams for multimodal information: tracking image data and passer's stats and classifying pass success and failure. After completing the classification, it provides a rationale by either calculating the relative contribution between the different modality data or providing more detailed contribution factors within the modality. The results of the experiment with 6,349 passes of data obtained from professional soccer games revealed that PassAI showed higher classification performance than state-of-the-art algorithms by>5% and could visualize the rationale of the pass success/failure for both tracking and stats data. These results highlight the importance of using multimodality data in the sports domain to increase the performance of the artificial intelligence algorithm and explainability of the outcomes.

Problem

• Classify soccer pass success using multimodal data.
• Explain pass outcomes with tracking and player stats.
• Enhance AI performance and explainability in sports analysis.

Innovation

• Multimodal data integration for pass analysis
• Explainable AI with tracking and stats data
• Enhanced classification performance and rationale visualization