Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models

2023-07-16

IEEE Transactions on Software Engineering

AI Summary

This study addresses the challenge of quantifying prediction uncertainty for large language models (LLMs) in safety-critical applications. We introduce the first systematic empirical framework to evaluate twelve uncertainty estimation methods—including confidence scores, entropy, Monte Carlo Dropout, and logit margin—across four major LLM families (Llama, GPT-2, OPT, Bloom) and six diverse tasks (four NLP benchmarks and two code generation datasets). Results demonstrate that uncertainty metrics effectively distinguish reliable from unreliable predictions, achieving AUC scores of 0.72–0.89; notably, they identify approximately 68% of defective programs in code generation. Crucially, we provide the first empirical evidence that uncertainty signals exhibit discriminative power for detecting non-factual outputs and hallucinations. Our work establishes a novel, interpretable, and deployable risk assessment paradigm for trustworthy LLMs, grounded in rigorous cross-model and cross-task evaluation.

Abstract

The recent performance leap of Large Language Models (LLMs) opens up new opportunities across numerous industrial applications and domains. However, erroneous generations, such as false predictions, misinformation, and hallucination made by LLMs, have also raised severe concerns for the trustworthiness of LLMs', especially in safety-, security- and reliability-sensitive scenarios, potentially hindering real-world adoptions. While uncertainty estimation has shown its potential for interpreting the prediction risks made by general machine learning (ML) models, little is known about whether and to what extent it can help explore an LLM's capabilities and counteract its undesired behavior. To bridge the gap, in this paper, we initiate an exploratory study on the risk assessment of LLMs from the lens of uncertainty. In particular, we experiment with twelve uncertainty estimation methods and four LLMs on four prominent natural language processing (NLP) tasks to investigate to what extent uncertainty estimation techniques could help characterize the prediction risks of LLMs. Our findings validate the effectiveness of uncertainty estimation for revealing LLMs' uncertain/non-factual predictions. In addition to general NLP tasks, we extensively conduct experiments with four LLMs for code generation on two datasets. We find that uncertainty estimation can potentially uncover buggy programs generated by LLMs. Insights from our study shed light on future design and development for reliable LLMs, facilitating further research toward enhancing the trustworthiness of LLMs.

Problem

• Uncertainty Estimation
• Large Language Models
• Reliability in Critical Applications

Innovation

• Uncertainty Estimation
• Large Language Models
• Code Generation