When Does Delegation Beat Majority? A Delegation-Based Aggregator for Multi-Sample LLM Inference

2026-06-06

AI Summary

This work addresses the suboptimal aggregation in existing majority voting methods for multi-sample large language model inference, which typically neglect semantic entropy and geometric signals in reasoning paths. The authors propose PPV, an unsupervised aggregation mechanism that uniquely integrates token-level semantic entropy with cosine similarity of reasoning embeddings to construct a stochastic delegation matrix, selecting the consensus answer via its stationary distribution. PPV innovatively introduces a dual-lever framework—determining both “when to delegate” and “whom to delegate to”—thereby transcending the limitations of conventional voting schemes. Evaluated on MMLU-Pro, the method achieves a 1.5 percentage point absolute gain in overall accuracy and a 2.24 percentage point improvement on non-trivial subsets (p ≈ 1.0 × 10⁻¹⁴), all without requiring labels or additional training.

Abstract

Majority voting over sampled answers is the dominant unsupervised aggregator for multi-sample LLM inference. We show that piping the signals every sample carries into a delegation-based aggregator (Propagational Proxy Voting, PPV) yields an unsupervised consensus rule that beats majority on MMLU-Pro by +1.5 pp overall and +2.24 pp on the non-trivial subset (paired McNemar p ~ 1.0e-14, n = 8,099). Majority discards two free signals every sample carries: within-group letter entropy and between-group reasoning geometry. PPV exposes two per-voter levers that consume exactly these signals: WHEN (how much weight a voter keeps on its own pick) and WHOM (how it splits the remainder across peers). We drive WHEN with letter entropy and WHOM with per-question-centered embedding cosine. The method needs no gold labels and no auxiliary training: per question, we partition 128 sampled generations into 16 groups, compute each group's letter-level semantic entropy and reasoning embedding centroid, and feed both into a stochastic delegation matrix whose stationary distribution selects the consensus answer. We walk through an example in which PPV overturns a clear 10-6 majority for the wrong letter: the 10-voter majority cluster is geometrically incoherent (mean within-cluster cosine -0.02) while the 6-voter minority is tight (+0.26), so propagated delegation mass concentrates on the minority's answer even though entropy alone would keep the majority ahead. We further report delegation strategies with negative results that constrain the design space for unsupervised LLM aggregation: no within-question ensemble of confidence modes closes the oracle gap.

Problem

• LLM inference
• unsupervised aggregation
• majority voting
• delegation-based consensus
• multi-sample reasoning

Innovation

• delegation-based aggregation
• semantic entropy
• reasoning geometry
• unsupervised consensus
• large language models