Localize-Then-Decide Guarantees for LLM Judgments

2026-08-26

AI Summary

为解决大语言模型评价输出质量时与人类判断一致性的挑战，提出了一种先定位后决策的框架，通过局部化优选短名单并基于校准后的置信度规则选择或弃权，从而保证高概率一致性。

Abstract

Large language models (LLMs) are increasingly used as evaluators to assess output quality and preference alignment, yet providing reliable guarantees of agreement with human judgments remains challenging. Recent work introduces confidence-thresholding methods that provide such guarantees for pairwise comparisons, relying on the assumption that higher estimated confidence implies lower disagreement risk with humans. However, this assumption can break down when the number of candidate responses increases, since distributing probability mass across many alternatives can distort confidence estimates. To address this issue, we propose a Localize-Then-Decide framework. First, conformal prediction localizes a small shortlist that contains the human-preferred response with high probability. Then, a calibrated confidence-based rule selectively chooses a single response from this shortlist or abstains. This design restores the monotonic relationship between confidence and disagreement risk and enables high-probability agreement guarantees. Experiments with multiple candidate sizes across several datasets and judge LLMs demonstrate that our framework consistently achieves higher guarantee success rates and substantially higher coverage than single-stage baselines.

Problem

• large language models
• human judgments
• confidence-thresholding
• candidate responses
• conformal prediction

Innovation

• Localize-Then-Decide
• conformal prediction
• confidence-based rule
• agreement guarantees