Minority Sentinel: When to Overturn Majority Voting in Multi-Agent LLM Debates

2026-06-28

AI Summary

This work addresses a critical limitation in multi-agent large language model (LLM) debate systems: highly correlated model errors systematically suppress correct minority opinions—a phenomenon termed the “minority truth” problem—thereby constraining collective reasoning performance. The study is the first to formally identify and quantify this issue, and proposes a lightweight meta-classifier that determines when to overturn majority-vote outcomes to recover correct minority answers, without relying on additional LLM calls. The method extracts multidimensional behavioral fingerprints from debate logs and trains a LightGBM classifier to predict beneficial vote reversals. Evaluated across six benchmark datasets with 20 random seeds, the approach achieves an 81.2% precision in identifying such reversals and yields consistent net gains in accuracy, significantly outperforming baselines such as LLM-as-Judge and effectively enhancing overall reasoning fidelity.

Abstract

Multi-Agent Debate (MAD) with Majority Voting is a dominant paradigm for improving LLM reasoning, yet its effectiveness rests on the Condorcet Jury Theorem's assumption of independent errors. Because contemporary LLMs share similar pretraining corpora, their errors are strongly correlated, causing the majority to systematically suppress correct minority opinions, a phenomenon we term Minority Truth. Through debates among three heterogeneous LLM agents on six benchmarks, we find that roughly one in four divergent cases has the minority holding the correct answer, yielding a 10-percentage-point theoretical recovery margin. We propose Minority Sentinel, a lightweight meta-classifier that extracts a multi-dimensional debate fingerprint from debate logs and trains a LightGBM model to decide when to overturn majority voting. Minority Sentinel achieves a stable Flip Precision of 81.2% with positive Net Gain across all six datasets and all 20 random seed trials, demonstrating that debate logs contain sufficient behavioral signals for a non-LLM classifier to reliably recover suppressed minorities without degrading system accuracy. The LLM-as-Judge baseline yields negative Net Gain despite higher recall, confirming that flip safety, not recovery volume, determines intervention value.

Problem

• Minority Truth
• Majority Voting
• Multi-Agent Debate
• Error Correlation
• LLM Reasoning

Innovation

• Minority Sentinel
• Multi-Agent Debate
• Majority Voting Overturn
• Debate Fingerprint
• LightGBM Meta-Classifier