Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation

2025-10-02

AI Summary

This paper addresses a core challenge in uncertainty estimation evaluation for natural language generation (NLG): inconsistent correctness annotations—particularly due to hallucinated content in LLM outputs—distorting method rankings. Methodologically, it abandons single proxy correctness metrics in favor of multiple risk-sensitive alternatives; employs multi-version LLM-as-a-judge to marginalize over quality assessments; and applies the Elo rating system for robust, cross-method ranking. Its key contribution is the first integration of risk-aware modeling, discriminative judging, and game-theoretic scoring into a unified, bias-mitigated, comparable, and robust evaluation framework. Experiments demonstrate substantial reduction in evaluation bias, yielding more reliable method rankings across question answering, structured generation, and out-of-distribution detection tasks—thereby enhancing scientific rigor and reproducibility in uncertainty estimation research.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Hallucinations are a common issue that undermine the reliability of large language models (LLMs). Recent studies have identified a specific subset of hallucinations, known as confabulations, which arise due to predictive uncertainty of LLMs. To detect confabulations, various methods for estimating predictive uncertainty in natural language generation (NLG) have been developed. These methods are typically evaluated by correlating uncertainty estimates with the correctness of generated text, with question-answering (QA) datasets serving as the standard benchmark. However, commonly used approximate correctness functions have substantial disagreement between each other and, consequently, in the ranking of the uncertainty estimation methods. This allows one to inflate the apparent performance of uncertainty estimation methods. We propose using several alternative risk indicators for risk correlation experiments that improve robustness of empirical assessment of UE algorithms for NLG. For QA tasks, we show that marginalizing over multiple LLM-as-a-judge variants leads to reducing the evaluation biases. Furthermore, we explore structured tasks as well as out of distribution and perturbation detection tasks which provide robust and controllable risk indicators. Finally, we propose to use an Elo rating of uncertainty estimation methods to give an objective summarization over extensive evaluation settings.

Problem

• Evaluating uncertainty estimation methods for detecting LLM hallucinations faces reliability issues
• Common correctness metrics show substantial disagreement in ranking uncertainty estimation methods
• Proposing robust evaluation frameworks using multiple risk indicators and Elo ratings

Innovation

• Using multiple LLM-as-a-judge variants for bias reduction
• Exploring structured tasks for robust risk indicators
• Applying Elo rating for objective method summarization