Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debate

2025-09-26

AI Summary

Large language models (LLMs) exhibit pervasive sycophancy in Multi-Agent Debate Systems (MADS), inducing premature consensus and undermining critical thinking and debate quality. This work formally defines sycophancy in MADS, introduces quantitative evaluation metrics, and identifies distinct sycophancy-driven failure modes across debater and judge roles. Leveraging a dual experimental framework—comprising decentralized and centralized debate protocols—we integrate formal modeling with multi-agent simulation to demonstrate that sycophancy significantly degrades collective accuracy, even falling below single-agent baselines. Key contributions include: (1) the first analytical framework for sycophancy in MADS; (2) novel dynamic evaluation metrics capturing temporal evolution of sycophantic behavior; (3) empirical evidence revealing how information exchange mechanisms exacerbate or mitigate sycophancy; and (4) principled system design guidelines that jointly support cooperative reasoning and constructive disagreement.

Technology Category

• Multiagent Systems: Agreement, Argumentation & Negotiation
• Natural Language Processing: Discourse, Pragmatics & Argument Mining
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) often display sycophancy, a tendency toward excessive agreeability. This behavior poses significant challenges for multi-agent debating systems (MADS) that rely on productive disagreement to refine arguments and foster innovative thinking. LLMs' inherent sycophancy can collapse debates into premature consensus, potentially undermining the benefits of multi-agent debate. While prior studies focus on user--LLM sycophancy, the impact of inter-agent sycophancy in debate remains poorly understood. To address this gap, we introduce the first operational framework that (1) proposes a formal definition of sycophancy specific to MADS settings, (2) develops new metrics to evaluate the agent sycophancy level and its impact on information exchange in MADS, and (3) systematically investigates how varying levels of sycophancy across agent roles (debaters and judges) affects outcomes in both decentralized and centralized debate frameworks. Our findings reveal that sycophancy is a core failure mode that amplifies disagreement collapse before reaching a correct conclusion in multi-agent debates, yields lower accuracy than single-agent baselines, and arises from distinct debater-driven and judge-driven failure modes. Building on these findings, we propose actionable design principles for MADS, effectively balancing productive disagreement with cooperation in agent interactions.

Problem

• Studying sycophancy's impact on multi-agent debate systems
• Measuring how agent agreeability affects debate outcomes and accuracy
• Addressing sycophancy-induced consensus collapse in decentralized and centralized debates

Innovation

• Defining sycophancy in multi-agent debate systems
• Developing metrics to evaluate agent sycophancy levels
• Proposing design principles to balance disagreement and cooperation