Transformer Guided Coevolution: Improved Team Formation in Multiagent Adversarial Games

2024-10-17

arXiv.org

AI Summary

This paper addresses the challenge of optimal team composition in multi-agent adversarial team games. Methodologically, it proposes a co-evolutionary deep reinforcement learning framework integrated with large language model (LLM)-driven optimization: (1) diverse individual policies are trained via co-evolutionary reinforcement learning to enhance behavioral heterogeneity; (2) BERTeam—a novel Transformer-based architecture incorporating masked language modeling (MLM)—is introduced as the first end-to-end model for jointly learning team composition and win-rate prediction. The key contribution lies in overcoming the limitations of conventional heuristic or population-based search methods, enabling generalizable and robust automated team formation. Evaluated on the Marine Capture-The-Flag environment, the approach significantly outperforms baselines such as MCAA, automatically discovers high-win-rate, non-trivial team structures, and demonstrates strong generalization to unseen opponents.

Technology Category

• Multiagent Systems: Adversarial Agents
• Humans and AI: Teamwork, Team formation
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We consider the problem of team selection within multiagent adversarial team games. We propose BERTeam, a novel algorithm that uses a transformer-based deep neural network with Masked Language Model training to select the best team of players from a trained population. We integrate this with coevolutionary deep reinforcement learning, which trains a diverse set of individual players to choose from. We test our algorithm in the multiagent adversarial game Marine Capture-The-Flag, and find that BERTeam learns non-trivial team compositions that perform well against unseen opponents. For this game, we find that BERTeam outperforms MCAA, an algorithm that similarly optimizes team selection.

Problem

• Multi-player games
• Optimal team composition
• Unknown opponents strategy

Innovation

• BERTeam Algorithm
• Transformer Deep Learning
• Co-evolutionary Deep Reinforcement Learning