Margin-Adaptive Confidence Ranking for Reliable LLM Judgement

2026-05-14

AI Summary

This work challenges the common assumption that the confidence scores of large language models (LLMs) monotonically correlate with the risk of disagreement among human judgments—a premise often violated in practice and lacking theoretical guarantees for generalization. To address this, the authors propose a margin-aware confidence ranking framework that explicitly models an LLM’s ability to distinguish between cases of human consensus and disagreement by simulating annotator diversity and leveraging margin-based ranking learning. The approach integrates sequential fixed-horizon testing to enhance judgment reliability and, for the first time, provides generalization error bounds for confidence estimators, revealing a fundamental trade-off governed by the margin. Guided by this insight, an adaptive training strategy is devised to reinforce monotonicity. Experiments demonstrate consistent improvements in confidence ranking accuracy and agreement with target human judgments across multiple datasets and evaluator models.

Abstract

Jung et al. (2025) introduce a hypothesis testing framework for guaranteeing agreement between large language models (LLMs) and human judgments, relying on the assumption that the model's estimated confidence is monotonic with respect to human-disagreement risk. In practice, however, this assumption may be violated, and the generalization behavior of the confidence estimator is not explicitly analyzed. We mitigate these issues by learning a dedicated confidence estimator instead of relying on heuristic confidence signals. Our approach leverages simulated annotator diversity and a margin-based ranking formulation to explicitly model how confidently an LLM distinguishes between human-agreement and human-disagreement cases. We further derive generalization guarantees for this estimator, revealing a margin-dependent trade-off that informs the design of an adaptive estimator training procedure. When integrated into fixed-sequence testing, the learned confidence estimator yields improved ranking accuracy and empirically strengthens the monotonic relationship between confidence and disagreement risk, leading to higher success rates in satisfying target agreement levels across multiple datasets and judge models.

Problem

• confidence estimation
• human disagreement
• monotonicity assumption
• generalization guarantee
• LLM reliability

Innovation

• confidence estimation
• margin-based ranking
• generalization guarantee
• annotator diversity simulation
• LLM-human agreement