Generalizable Agent Modeling for Agent Collaboration-Competition Adaptation with Multi-Retrieval and Dynamic Generation

2025-06-20

AI Summary

This paper addresses the challenge of zero-shot generalization for a single agent operating in multi-agent systems with unknown teammates and opponents. To tackle this, we propose the Adaptive Collaboration-Competition Assessment (ACCA) paradigm—a novel evaluation framework supporting cross-task, cross-environment, and cross-interaction zero-shot transfer. Methodologically, we introduce the Multi-Retrieval Dynamic Generation (MRDG) architecture, which integrates variable-scale team positional encoding, a lightweight hypernetwork, and a cross-agent viewpoint alignment module to enable unified modeling and dynamic response to heterogeneous partner behaviors. Evaluated on SMAC, Overcooked-AI, and Melting Pot benchmarks, ACCA achieves significant improvements in collaborative robustness and competitive win rates, consistently outperforming state-of-the-art zero-shot MARL and ad-hoc teamwork baselines. Our work establishes a scalable, principled assessment framework and modeling paradigm for generalization in multi-agent reinforcement learning.

Technology Category

• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Adapting a single agent to a new multi-agent system brings challenges, necessitating adjustments across various tasks, environments, and interactions with unknown teammates and opponents. Addressing this challenge is highly complex, and researchers have proposed two simplified scenarios, Multi-agent reinforcement learning for zero-shot learning and Ad-Hoc Teamwork. Building on these foundations, we propose a more comprehensive setting, Agent Collaborative-Competitive Adaptation (ACCA), which evaluates an agent to generalize across diverse scenarios, tasks, and interactions with both unfamiliar opponents and teammates. In ACCA, agents adjust to task and environmental changes, collaborate with unseen teammates, and compete against unknown opponents. We introduce a new modeling approach, Multi-Retrieval and Dynamic Generation (MRDG), that effectively models both teammates and opponents using their behavioral trajectories. This method incorporates a positional encoder for varying team sizes and a hypernetwork module to boost agents'learning and adaptive capabilities. Additionally, a viewpoint alignment module harmonizes the observational perspectives of retrieved teammates and opponents with the learning agent. Extensive tests in benchmark scenarios like SMAC, Overcooked-AI, and Melting Pot show that MRDG significantly improves robust collaboration and competition with unseen teammates and opponents, surpassing established baselines. Our code is available at: https://github.com/vcis-wangchenxu/MRDG.git

Problem

• Adapting single agents to multi-agent systems with unknown teammates and opponents
• Generalizing agent performance across diverse scenarios and tasks
• Enhancing collaboration and competition with unseen agents using behavioral modeling

Innovation

• Multi-Retrieval and Dynamic Generation modeling
• Positional encoder for varying team sizes
• Viewpoint alignment module for observational harmonization