Think Through Uncertainty: Improving Long-Form Generation Factuality via Reasoning Calibration

2026-04-13

AI Summary

Large language models often suffer from hallucinations in long-form text generation and lack the ability to assess the credibility of fine-grained factual claims. To address this, this work proposes CURE, a novel framework that enables statement-level uncertainty reasoning and confidence calibration for the first time. By integrating a claim-aware reasoning protocol with multi-stage alignment and reinforcement learning, CURE allows the model to generate calibrated confidence scores alongside each atomic claim during decoding and to selectively abstain from producing low-confidence content at inference time. Experimental results demonstrate that CURE significantly outperforms existing baselines across four long-text factuality benchmarks, achieving up to a 39.9% absolute improvement in claim-level accuracy on biography generation and a 16.0% gain in AUROC on FactBench.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Large language models (LLMs) often hallucinate in long-form generation. Existing approaches mainly improve factuality through post-hoc revision or reinforcement learning (RL) with correctness-based rewards, but they do not teach the model to estimate which parts of its generation are reliable. As a result, models may still state incorrect claims confidently in their responses. Recent advances in reasoning have significantly improved LLM performance, and have been leveraged to estimate confidence by incorporating calibration into RL objectives. However, existing approaches remain limited to a single scalar confidence for the entire response, which is insufficient for long-form generation where uncertainty varies across individual claims. To mitigate this problem, we propose CURE, a framework that improves long-form factuality by teaching LLMs to reason about uncertainty at the claim level. We first introduce a Claim-Aware Reasoning Protocol, which structures outputs into atomic claims paired with explicit confidence estimates. We then develop a multi-stage training pipeline that aligns model confidence with claims' correctness and then optimizes on factuality. The resulting calibrated confidence further enables selective prediction, allowing the model to abstain from uncertain claims at inference time. Experiments on four long-form factuality benchmarks show that CURE consistently improves factual accuracy over competitive supervised and RL baselines, while maintaining factual recall. In particular, it improves claim-level accuracy by up to 39.9% on Biography generation. These gains are accompanied by improved calibration, as reflected by a 16.0% increase in AUROC on FactBench.

Problem

• hallucination
• factuality
• uncertainty calibration
• long-form generation
• claim-level confidence

Innovation

• reasoning calibration
• claim-level uncertainty
• long-form factuality
• selective prediction
• confidence alignment