Adaptive Coopetition: Leveraging Coarse Verifier Signals for Resilient Multi-Agent LLM Reasoning

2025-10-20

AI Summary

Existing reasoning-time strategies exhibit critical limitations: self-correction tends to reinforce initial biases, multi-agent collaboration (MAC) often suffers from insufficient coordination leading to collective errors, and high-accuracy verifiers require extensive human annotations. This paper proposes AdCo, the first framework to introduce a UCB-based adaptive “coopetition” mechanism—dynamically balancing cooperation and competition—into multi-agent LLM reasoning. AdCo leverages only coarse-grained verification signals to guide uncertainty-aware exploration and enhance trajectory diversity. Its methodology integrates multi-agent collaborative reasoning, iterative refinement, reasoning trajectory analysis, and knowledge diversity modeling. Evaluated on multiple mathematical reasoning benchmarks, AdCo achieves a 20% relative performance gain over state-of-the-art baselines while demonstrating robustness across varying sample sizes and configuration settings.

Technology Category

• Multiagent Systems: Adversarial Agents
• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Inference-time computation is a critical yet challenging paradigm for enhancing the reasoning performance of large language models (LLMs). While existing strategies improve reasoning stability and consistency, they suffer from notable limitations: self-correction often reinforces the model's initial biases, and Multi-Agent Collaboration (MAC) often fails due to the lack of efficient coordination mechanisms, leading to collective errors. Although high-performing verifiers can detect reasoning errors, making them reliable requires substantial training. To address these challenges, we introduce a novel inference-time framework, Adaptive Coopetition (AdCo), in which LLM agents utilize an adaptive, UCB-based "coopetition" mechanism. At each round, agents leverage coarse verifier signals to determine whether to collaborate or compete, and iteratively refine their reasoning based on peer feedback. Without relying on high-performance verifiers, our adaptive strategy achieves significant performance gains on mathematical reasoning benchmarks, yielding a 20% relative improvement over baselines on the more challenging dataset. Our approach remains robust and consistent in terms of accuracy under different sample sizes and configurations. This adaptive, signal-guided "coopetition" framework enhances reasoning robustness by leveraging both model knowledge diversity and reasoning trace measures, while also promoting uncertainty-driven exploration, especially when participants have comparable capabilities. From this perspective, our work offers a fresh lens on inference-time computation and paves the way for more resilient multi-agent LLM systems. Our code is available at: https://github.com/AdCo-Research/adaptive-coopetition.

Problem

• Enhancing LLM reasoning stability without reinforcing initial biases
• Addressing coordination failures in multi-agent collaboration systems
• Improving reasoning robustness using coarse verifier signals adaptively

Innovation

• Adaptive coopetition mechanism using UCB-based strategy
• Leveraging coarse verifier signals for collaboration decisions
• Iterative reasoning refinement through peer feedback cycles