TIGER-MARL: Enhancing Multi-Agent Reinforcement Learning with Temporal Information through Graph-based Embeddings and Representations

2025-11-11

AI Summary

Existing multi-agent reinforcement learning (MARL) methods typically employ static or frame-wise updated relational graphs, failing to capture the natural temporal evolution of interaction structures during agent collaboration. To address this, we propose the Dynamic Temporal Graph Reinforcement Learning framework (DTG-RL), the first to explicitly model the continuous temporal evolution of collaborative relationships. DTG-RL introduces a learnable dynamic graph construction module to capture structural changes in interactions and integrates a temporal attention encoder to generate time-aware agent embeddings, enabling joint representation learning across both structural and temporal dimensions. Evaluated on two high-cooperation benchmarks—StarCraft II and Google Research Football—DTG-RL significantly outperforms state-of-the-art value-decomposition and graph-based MARL methods in both task performance and sample efficiency. Ablation studies confirm the critical contributions of dynamic graph modeling and temporal attention.

Technology Category

• Multiagent Systems: Multiagent Learning
• Machine Learning: Graph-based Machine Learning
• Planning, Routing, and Scheduling: Temporal Planning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this paper, we propose capturing and utilizing extit{Temporal Information through Graph-based Embeddings and Representations} or extbf{TIGER} to enhance multi-agent reinforcement learning (MARL). We explicitly model how inter-agent coordination structures evolve over time. While most MARL approaches rely on static or per-step relational graphs, they overlook the temporal evolution of interactions that naturally arise as agents adapt, move, or reorganize cooperation strategies. Capturing such evolving dependencies is key to achieving robust and adaptive coordination. To this end, TIGER constructs dynamic temporal graphs of MARL agents, connecting their current and historical interactions. It then employs a temporal attention-based encoder to aggregate information across these structural and temporal neighborhoods, yielding time-aware agent embeddings that guide cooperative policy learning. Through extensive experiments on two coordination-intensive benchmarks, we show that TIGER consistently outperforms diverse value-decomposition and graph-based MARL baselines in task performance and sample efficiency. Furthermore, we conduct comprehensive ablation studies to isolate the impact of key design parameters in TIGER, revealing how structural and temporal factors can jointly shape effective policy learning in MARL. All codes can be found here: https://github.com/Nikunj-Gupta/tiger-marl.

Problem

• Modeling evolving inter-agent coordination structures over time
• Capturing temporal evolution of interactions in multi-agent systems
• Enhancing cooperative policy learning with time-aware agent embeddings

Innovation

• Using dynamic temporal graphs to model agent interactions
• Employing temporal attention for time-aware embeddings
• Enhancing multi-agent reinforcement learning with evolving dependencies