Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis

2025-10-01

AI Summary

This study addresses the limited interpretability and generalizability of player-level agent modeling in highly coupled team sports (e.g., football) under heterogeneous data. We propose the Expandable Decision-Making States (EDMS) framework: (1) semantic-enriched state representation—encoding pressure, passing lanes, scoring potential, and pass feasibility—to map high-dimensional observations onto interpretable tactical concepts; (2) a relation-aware action masking mechanism that differentiates decision spaces for ball-possessing versus non-possessing players. EDMS integrates multi-agent reinforcement learning, relational feature extraction, and semantic state expansion to ensure rule consistency and cross-dataset compatibility. Experiments demonstrate that EDMS significantly reduces action prediction loss and TD error, accurately identifies critical tactical patterns (e.g., rapid counterattacks, defensive breakthroughs), and supports reproducible research across open-source and commercial multi-source datasets.

Technology Category

• Multiagent Systems: Teamwork
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Invasion team sports such as soccer produce a high-dimensional, strongly coupled state space as many players continuously interact on a shared field, challenging quantitative tactical analysis. Traditional rule-based analyses are intuitive, while modern predictive machine learning models often perform pattern-matching without explicit agent representations. The problem we address is how to build player-level agent models from data, whose learned values and policies are both tactically interpretable and robust across heterogeneous data sources. Here, we propose Expandable Decision-Making States (EDMS), a semantically enriched state representation that augments raw positions and velocities with relational variables (e.g., scoring of space, pass, and score), combined with an action-masking scheme that gives on-ball and off-ball agents distinct decision sets. Compared to prior work, EDMS maps learned value functions and action policies to human-interpretable tactical concepts (e.g., marking pressure, passing lanes, ball accessibility) instead of raw coordinate features, and aligns agent choices with the rules of play. In the experiments, EDMS with action masking consistently reduced both action-prediction loss and temporal-difference (TD) error compared to the baseline. Qualitative case studies and Q-value visualizations further indicate that EDMS highlights high-risk, high-reward tactical patterns (e.g., fast counterattacks and defensive breakthroughs). We also integrated our approach into an open-source library and demonstrated compatibility with multiple commercial and open datasets, enabling cross-provider evaluation and reproducible experiments.

Problem

• Modeling player-level agents with interpretable tactical policies
• Representing complex soccer interactions with relational state variables
• Enabling robust tactical analysis across heterogeneous data sources

Innovation

• EDMS augments raw data with relational tactical variables
• Action masking provides distinct decision sets for agents
• EDMS maps learned functions to human-interpretable tactical concepts