Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
Traditional multi-agent debate (MAD) often fails to effectively enhance large language model performance due to homogeneous agents and uniform belief updating, sometimes even underperforming simple majority voting. This work proposes an improved framework that better mirrors human-like negotiation mechanisms: it employs diversity-aware initialization to increase the prior probability of correct hypotheses and introduces explicit confidence calibration in agent communication, coupled with a confidence-weighted belief update rule to systematically guide the debate toward the correct answer. Theoretical analysis and extensive experiments across six reasoning-based question-answering benchmarks demonstrate that the proposed approach significantly outperforms both conventional MAD and majority voting, thereby substantially improving the accuracy and reliability of multi-agent debates.