Adversarial Coevolutionary Illumination with Generational Adversarial MAP-Elites

2025-05-10

AI Summary

Traditional quality diversity (QD) algorithms often evolve only one agent side in adversarial multi-agent settings, leading to insufficient coverage of the behavioral space. To address this, we propose Generational Adversarial MAP-Elites (GAME), the first QD framework enabling co-evolution of both adversaries. GAME employs a generational alternating adversarial mechanism to simultaneously illuminate the strategy spaces of both “red” and “blue” agents. It introduces end-to-end video-based behavioral embeddings, eliminating hand-crafted feature engineering, and integrates restart-based reinitialization with neutral mutations to enhance evolutionary openness while preserving critical behavioral transitions. Experiments demonstrate that GAME significantly improves both strategic diversity and performance balance. It is the first method to achieve high-quality, high-coverage bidirectional illumination of strategy spaces in adversarial games, establishing a novel paradigm for studying long-term co-evolutionary dynamics among intelligent agents.

Technology Category

• Multiagent Systems: Adversarial Agents
• Game Theory and Economic Paradigms: Adversarial Learning
• Search and Optimization: Adversarial Search

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Unlike traditional optimization algorithms focusing on finding a single optimal solution, Quality-Diversity (QD) algorithms illuminate a search space by finding high-performing solutions that cover a specified behavior space. However, tackling adversarial problems is more challenging due to the behavioral interdependence between opposing sides. Most applications of QD algorithms to these problems evolve only one side, thus reducing illumination coverage. In this paper, we propose a new QD algorithm, Generational Adversarial MAP-Elites (GAME), which coevolves solutions by alternating sides through a sequence of generations. Combining GAME with vision embedding models enables the algorithm to directly work from videos of behaviors instead of handcrafted descriptors. Some key findings are that (1) emerging evolutionary dynamics sometimes resemble an arms race, (2) starting each generation from scratch increases open-endedness, and (3) keeping neutral mutations preserves stepping stones that seem necessary to reach the highest performance. In conclusion, the results demonstrate that GAME can successfully illuminate an adversarial multi-agent game, opening up interesting future directions in understanding the emergence of open-ended coevolution.

Problem

• Addresses adversarial coevolution in Quality-Diversity algorithms
• Enhances illumination coverage via generational adversarial coevolution
• Enables direct behavior analysis from videos using vision embedding

Innovation

• Generational Adversarial MAP-Elites coevolves solutions
• Combines with vision embedding for video-based behavior
• Neutral mutations preserve stepping stones