RUMAD: Reinforcement-Unifying Multi-Agent Debate

2026-02-27

AI Summary

Existing multi-agent debate approaches struggle to simultaneously achieve accuracy, consensus formation, and computational efficiency, while static communication topologies lack adaptability and external coordination often introduces bias. This work proposes the first reinforcement learning–based framework for dynamic communication topology control, which adaptively reconfigures agent interactions through content-agnostic, high-order debate dynamics observation and a dual-threshold activation mechanism, thereby preserving neutrality. Employing a multi-objective reward function—balancing solution quality, cohesion, and efficiency—and a PPO-based controller, the method reduces token consumption by over 80% while improving accuracy on MMLU, GSM8K, and GPQA benchmarks. Notably, training solely on MMLU enables strong zero-shot generalization across diverse tasks.

Technology Category

• Multiagent Systems: Agent Communication
• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Multi-agent debate (MAD) systems leverage collective intelligence to enhance reasoning capabilities, yet existing approaches struggle to simultaneously optimize accuracy, consensus formation, and computational efficiency. Static topology methods lack adaptability to task complexity variations, while external LLM-based coordination risks introducing privileged knowledge that compromises debate neutrality. This work presents RUMAD (Reinforcement-Unifying Multi-Agent Debate), a novel framework that formulates dynamic communication topology control in MAD as a reinforcement learning (RL) problem. RUMAD employs a content-agnostic observation scheme that captures high-level debate dynamics avoiding access to raw agent reasoning content. RUMAD uses a multi-objective reward to model solution quality, cohesion and efficiency. A PPO-trained controller dynamically adjusts edge weights in the communication graph, while a dual-threshold mechanism enables fine-grained control over both agent activation and information visibility. Experimental evaluation across MMLU, GSM8K, and GPQA benchmarks demonstrates that RUMAD achieves substantial efficiency gains, reducing token costs by over 80\%, while still improving reasoning accuracy compared to single LLM model and multiple MAD baselines. Notably, RUMAD trained exclusively on MMLU exhibits robust zero-shot generalization to out-of-domain (OOD) tasks, indicating that the learned communication strategies capture task-independent principles of effective multi-agent coordination. These results establish RUMAD as a efficient and robust approach for deploying multi-agent reasoning application with practical resource constraints.

Problem

• multi-agent debate
• communication topology
• computational efficiency
• consensus formation
• reasoning accuracy

Innovation

• multi-agent debate
• reinforcement learning
• dynamic communication topology
• content-agnostic observation
• zero-shot generalization