L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning

2026-07-10

AI Summary

This study addresses the lack of systematic evaluation of multi-agent debate frameworks in knowledge-intensive legal reasoning tasks. The authors propose a Legal Multi-Agent Debate (L-MAD) framework that assigns expert roles to agents and systematically investigates the impact of different debate structures and aggregation strategies on legal textual entailment. Their analysis reveals, for the first time, the mechanisms by which agent population size and number of debate rounds influence reasoning performance, identifying a phenomenon termed “excessive negotiation drift” and delineating safety boundaries for collaborative systems. Experimental results demonstrate that L-MAD improves accuracy by up to 8% over strong single-agent baselines; increasing the number of agents reduces inconsistency and enhances performance, yet excessive debate rounds lead to performance degradation.

Abstract

While multi-agent debate (MAD) frameworks have shown significant potential in general reasoning, their effectiveness in highly structured, knowledge-heavy legal domains remains under-explored. In this work, we introduce the Legal Multi-Agent Debate (L-MAD) framework to systematically evaluate different debate structures and aggregation methods within Legal Textual Entailment. By assigning distinct expert personas to multiple agents, L-MAD improves upon strong single-agent baselines by up to 8\%. Furthermore, analyzing how debate scales reveals a clear trade-off: increasing the agent population reduces inconsistency and improves accuracy, whereas extending discussion rounds induces a detrimental \textit{over-deliberation drift} where agents reinforce each other's mistakes. Ultimately, our findings outline the practical boundaries and safety margins of deploying collaborative multi-agent systems in high-stakes legal reasoning environments.

Problem

• multi-agent debate
• legal reasoning
• Legal Textual Entailment
• expert personas
• over-deliberation drift

Innovation

• multi-agent debate
• legal reasoning
• expert personas
• over-deliberation drift
• systematic evaluation