Understanding the Relationship between Prompts and Response Uncertainty in Large Language Models

Unknown Date

AI Summary

This work addresses the reliability deficiency of large language models (LLMs) in high-stakes domains (e.g., healthcare) by quantifying the relationship between response uncertainty and input prompt informativeness. We propose a “prompt–response” conceptual model that formalizes response uncertainty as epistemic uncertainty and establishes an interpretable, decaying functional relationship between prompt informativeness and uncertainty. Methodologically, we integrate pretrained implicit learning of latent conceptual reasoning, information-theoretic measures, and an empirical analytical framework, validated systematically across multiple real-world datasets spanning diverse domains. Experiments demonstrate that increasing prompt informativeness significantly reduces response uncertainty. Our approach enables quantifiable, attribution-aware confidence estimation for high-risk applications, thereby enhancing the interpretability and controllability of LLM deployment in safety-critical settings.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) are widely used in decision-making, but their reliability, especially in critical tasks like healthcare, is not well-established. Therefore, understanding how LLMs reason and make decisions is crucial for their safe deployment. This paper investigates how the uncertainty of responses generated by LLMs relates to the information provided in the input prompt. Leveraging the insight that LLMs learn to infer latent concepts during pretraining, we propose a prompt-response concept model that explains how LLMs generate responses and helps understand the relationship between prompts and response uncertainty. We show that the uncertainty decreases as the prompt's informativeness increases, similar to epistemic uncertainty. Our detailed experimental results on real-world datasets validate our proposed model.

Problem

• Prompts affect response uncertainty
• Understanding LLMs' decision-making process
• Improving reliability in critical tasks

Innovation

• Prompt-response concept model
• Decreases uncertainty with informativeness
• Validated on real-world datasets