Coordination Matters: Evaluation of Cooperative Multi-Agent Reinforcement Learning

2026-05-07

AI Summary

Current evaluation practices in cooperative multi-agent reinforcement learning (MARL) rely excessively on aggregate metrics such as return, which often obscure the underlying coordination mechanisms—particularly in scenarios where tasks, agents, and assignment combinations scale combinatorially. This work proposes a coordination-centric evaluation paradigm and introduces STAT, a controlled task-allocation benchmark that systematically varies the number of agents, task complexity, and environment size under fixed observation and rule constraints. Using STAT, the study conducts a cross-spectrum analysis of coordination behaviors across six representative value-based MARL methods with varying degrees of centralization. For the first time, coordination mechanism analysis is integrated into MARL evaluation, revealing hidden differences in redundant allocations, allocation diversity, and task efficiency even when returns are identical. The findings demonstrate that return alone is insufficient to characterize system performance, establishing coordination-aware evaluation as a critical complement to cooperative MARL benchmarking.

Abstract

Cooperative multi-agent reinforcement learning (MARL) benchmarks commonly emphasize aggregate outcomes such as return, success rate, or completion time. While essential, these metrics often fail to reveal how agents coordinate, particularly in settings where agents, tasks, and joint assignment choices scale combinatorially. We propose a coordination-aware evaluation perspective that supplements return with process-level diagnostics. We instantiate this perspective using STAT, a controlled commitment-constrained spatial task-allocation testbed that systematically varies agents, tasks, and environment size while holding observation access and task rules fixed. We evaluate six representative value-based MARL methods across varying levels of centralization. Our results show that similar return trends can reflect distinct coordination mechanisms, including differences in redundant assignment, assignment diversity, and task-completion efficiency. We find that in commitment-constrained task allocation, performance under scale is shaped not only by nominal action-space size, but also by assignment pressure, sparse decision opportunities, and redundant choices among interdependent agents. Our findings motivate coordination-aware evaluation as a necessary complement to return-based benchmarking for cooperative MARL.

Problem

• cooperative multi-agent reinforcement learning
• coordination
• evaluation
• task allocation
• scalability

Innovation

• coordination-aware evaluation
• multi-agent reinforcement learning
• task allocation
• process-level diagnostics
• commitment constraints