SpanUQ: Span-Level Uncertainty Quantification for Large Language Model Generation

2026-07-06

AI Summary

This work addresses the limitations of existing uncertainty estimation methods for large language models, which lack semantic coherence at the token level and struggle to localize errors at the sequence level. To this end, the paper introduces the novel task of span-level uncertainty estimation (SLUE) and proposes SpanUQ, a lightweight probing framework that extracts uncertainties for semantically coherent text spans via a single forward pass from model hidden states. SpanUQ integrates a DETR-style span decoder, models uncertainty with a Beta mixture distribution, and employs joint training through Beta negative log-likelihood regression and contrastive ranking, enhanced by multi-sample inference-based knowledge distillation. The authors also construct SpanUQ-BENCH, the first large-scale evaluation benchmark featuring soft labels. Experiments demonstrate that SpanUQ consistently outperforms baselines across five prominent large language models, achieving up to 10–20× faster inference and a span detection F1 score of 0.910.

Abstract

Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self-refinement in LLM generation. However, existing approaches operate at suboptimal granularities: token-level scores lack semantic coherence, while sequence-level scores fail to localize errors. We formalize Span-Level Uncertainty Estimation (SLUE), a new task that targets the natural granularity for uncertainty: semantically coherent text spans, each conveying a single assessable unit of meaning. To address this task, we introduce SPANUQ, a lightweight probe that distills the uncertainty knowledge from expensive multi-sample inference into a single forward pass over LLM hidden states. SPANUQ employs a DETR-style span decoder to simultaneously detect spans and estimate their uncertainty via a Mixture of Beta distribution, trained with a principled combination of Beta NLL regression and contrastive ranking objectives. We construct SPANUQ-BENCH, the first span-level uncertainty benchmark comprising 20K prompts, 293K annotated spans, and continuous soft labels derived from multi-sample claim verification. Experiments on five LLM backbones show that SPANUQ consistently achieves the best span-level uncertainty quality, outperforming the strongest probe baseline and all sampling-based methods while being 10-20x faster. Its DETR-based span detector attains 0.910 F1, surpassing the best heuristic by 39.4%, enabling precise error localization that sequence-level methods cannot provide. The framework generalizes across five LLMs spanning two model families.

Problem

• uncertainty quantification
• large language models
• span-level
• error localization
• semantic coherence

Innovation

• Span-Level Uncertainty
• Uncertainty Quantification
• Large Language Models
• DETR-style Decoder
• Beta Mixture