Multi-Agent LLMs Fail to Explore Each Other

2026-07-13

AI Summary

This work addresses the suboptimal coordination and high regret often observed in multi-agent large language models due to insufficient exploration during interaction. It formalizes multi-agent exploration for the first time as a partially observable stochastic game and introduces MACE, a lightweight framework that explicitly promotes exploration through structured peer selection. Theoretical analysis demonstrates that the value of exploration increases with agent diversity. Empirical results validate that MACE significantly enhances exploratory behavior under both contextual and parametric diversity settings, leading to substantial improvements in downstream task performance across multiple benchmarks.

Abstract

Exploration is essential for reliable autonomy in multi-agent systems, yet it remains unclear whether large language model (LLM) agents can explore effectively when interacting with one another. We show that modern LLM agents fail to do so, often exhibiting myopic and polarized interaction patterns that lead to suboptimal coordination and increased regret. We formalize this challenge as the Multi-Agent Exploration problem, modeling it as a partially observable stochastic game (POSG) problem in which agents must probe peers to infer their capabilities and identify effective interaction strategies. To address this, we introduce Multi- Agent Contextual Exploration (MACE), a lightweight framework that explicitly promotes exploration through structured peer selection. Across both contextual and parametric diversity settings, MACE substantially improves exploration behavior and downstream task performance. We further show theoretically that the value of exploration increases with agent diversity. Overall, our results highlight a fundamental limitation of current LLM agents and underscore the importance of explicitly guided exploration for reliable multi-agent autonomy. Code will be released in https://github.com/deeplearning-wisc/mace

Problem

• Multi-Agent LLMs
• Exploration
• Coordination
• Partially Observable Stochastic Game
• Agent Diversity

Innovation

• multi-agent exploration
• large language models
• partially observable stochastic game
• MACE
• agent diversity