Mixture of Debaters: Learn to Debate at Architectural Level in Multi-Agent Reasoning

2026-06-28

AI Summary

Existing multi-agent debate frameworks rely on static architectures, incurring high computational overhead and lacking flexibility in dynamically adjusting agent roles and coordination mechanisms. This work proposes MoD, a unified self-debate framework based on a Mixture-of-Experts (MoE) architecture that simulates diverse debating behaviors within a single model. MoD decouples role assignment from process control via a dual-routing mechanism and employs momentum-based switching for smooth expert selection. Debate personas are encapsulated as lightweight expert modules, eliminating inter-agent communication costs. Experimental results demonstrate that MoD significantly outperforms both single-model baselines and conventional multi-agent systems across multimodal benchmarks, achieving higher accuracy while reducing inference latency by 3.7× and token consumption by 87%.

Abstract

Existing multi-agent debate frameworks suffer from two critical limitations: they rely on static architectures where agent roles and coordination patterns are fixed at design time, and they require instantiating multiple model copies, incurring substantial computational overhead. We propose Mixture of Debaters (MoD), a unified framework that enables dynamic self-debate within a single model by leveraging the Mixture-of-Experts paradigm. We address three key challenges in adapting MoE for dialectical reasoning: (1) dual-routing that decouples role allocation from process flow, dynamically determining when to debate versus when to synthesize; (2) momentum switching that smooths token-level routing with local context, reducing expert-switch jitter; and (3) unified self-debate that encapsulates diverse debating personas into lightweight expert modules, eliminating inter-agent communication while preserving behavioral diversity. Extensive experiments on multimodal benchmarks demonstrate that MoD outperforms both single-model baselines and conventional multi-agent systems, achieving superior accuracy with 3.7x lower latency and 87% reduction in token consumption.The source code can be accessed at https://github.com/YongLD/MoD.

Problem

• multi-agent debate
• static architecture
• computational overhead
• Mixture-of-Experts
• dialectical reasoning

Innovation

• Mixture-of-Experts
• multi-agent debate
• dynamic self-debate
• dual-routing
• unified reasoning framework