Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoning

2025-04-07

AI Summary

Multi-agent collaboration enhances large language models’ (LLMs) reasoning capabilities but incurs high computational overhead due to iterative interactions, and enforcing debate on low-difficulty queries risks error propagation. To address this, we propose the Confidence-Adaptive Debating (CAD) framework—the first to dynamically trigger multi-agent interaction based on response confidence: collaboration initiates only when a single agent’s confidence falls below a threshold, eliminating redundant debates; confidence-weighted aggregation and iterative refinement further suppress error propagation. CAD integrates four core components: an LLM-based multi-agent architecture, response confidence estimation, dynamic scheduling, and output optimization. Experiments demonstrate that CAD matches or surpasses state-of-the-art debating methods in accuracy while reducing inference latency by up to 42%, significantly improving the efficiency–accuracy trade-off.

Technology Category

• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Multiagent collaboration has emerged as a promising framework for enhancing the reasoning capabilities of large language models (LLMs). While this approach improves reasoning capability, it incurs substantial computational overhead due to iterative agent interactions. Furthermore, engaging in debates for queries that do not necessitate collaboration amplifies the risk of error generation. To address these challenges, we propose Debate Only When Necessary (DOWN), an adaptive multiagent debate framework that selectively activates the debate process based on the confidence score of the agent's initial response. For queries where debate is triggered, agents refine their outputs using responses from participating agents and their confidence scores. Experimental results demonstrate that this mechanism significantly improves efficiency while maintaining or even surpassing the performance of existing multiagent debate systems. We also find that confidence-guided debate mitigates error propagation and enhances the selective incorporation of reliable responses. These results establish DOWN as an optimization strategy for efficient and effective multiagent reasoning, facilitating the practical deployment of LLM-based collaboration.

Problem

• Reducing computational overhead in multiagent LLM reasoning
• Minimizing error generation in unnecessary agent debates
• Optimizing debate activation via confidence-based selection

Innovation

• Adaptive multiagent debate framework
• Confidence score triggers debate
• Refines outputs using agent responses