Sequence Modeling for N-Agent Ad Hoc Teamwork

2025-06-05

AI Summary

This paper addresses the N-Agent Heterogeneous Teamwork (NAHT) problem in multi-agent reinforcement learning—efficient collaboration among agents with no prior coordination, partial observability, and dynamically unknown numbers and types of teammates. We propose the first centralized Transformer-based NAHT framework, departing from independent learning paradigms by directly modeling cross-agent historical interaction sequences. Integrating centralized training with decentralized execution (CTDE) and multi-agent POMDP formalism, our approach achieves strong generalization without auxiliary tasks or explicit role inference. Evaluated on the StarCraft II benchmark, it improves sample efficiency by 37% over POAM and increases collaboration accuracy on unseen teammate types by 21%. This work provides the first empirical validation of sequential centralized modeling for dynamic, heterogeneous team coordination, demonstrating both effectiveness and superior generalization in open-team collaborative settings.

Technology Category

• Multiagent Systems: Teamwork
• Humans and AI: Human-AI Collaboration / Human-AI Teaming
• Game Theory and Economic Paradigms: Coordination and Collaboration

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

N-agent ad hoc teamwork (NAHT) is a newly introduced challenge in multi-agent reinforcement learning, where controlled subteams of varying sizes must dynamically collaborate with varying numbers and types of unknown teammates without pre-coordination. The existing learning algorithm (POAM) considers only independent learning for its flexibility in dealing with a changing number of agents. However, independent learning fails to fully capture the inter-agent dynamics essential for effective collaboration. Based on our observation that transformers deal effectively with sequences with varying lengths and have been shown to be highly effective for a variety of machine learning problems, this work introduces a centralized, transformer-based method for N-agent ad hoc teamwork. Our proposed approach incorporates historical observations and actions of all controlled agents, enabling optimal responses to diverse and unseen teammates in partially observable environments. Empirical evaluation on a StarCraft II task demonstrates that MAT-NAHT outperforms POAM, achieving superior sample efficiency and generalization, without auxiliary agent-modeling objectives.

Problem

• Dynamic collaboration with unknown teammates in NAHT
• Capturing inter-agent dynamics for effective teamwork
• Optimal responses in partially observable environments

Innovation

• Transformer-based centralized method for NAHT
• Incorporates historical observations and actions
• Enables optimal responses to unseen teammates