Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing

2025-07-11

AI Summary

Large language models (LLMs) frequently generate erroneous or questionable content with unwarranted confidence, undermining their credibility and effectiveness in human-AI collaboration. Current NLP research overlooks the pragmatic nuance of human uncertainty expression and exhibits systematic biases in how uncertainty is represented in training data. Method: We introduce “anthropomorphic uncertainty”—a framework wherein models emulate human-like, context-sensitive, and tunably personalized uncertainty expression. Drawing on linguistic theory and empirical studies of human communication, we systematically analyze existing approaches to verbalized uncertainty modeling in NLP, identifying critical issues including data bias and interaction mismatch. Contribution/Results: We propose a novel uncertainty expression framework that jointly optimizes linguistic authenticity, user adaptability, and interpretability. This framework establishes a new paradigm for building trustworthy, collaborative human-AI interfaces grounded in cognitively and pragmatically plausible uncertainty communication.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Natural Language Processing: Safety and Robustness
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Human users increasingly rely on natural language interactions with large language models (LLMs) in order to receive help on a large variety of tasks and problems. However, the trustworthiness and perceived legitimacy of LLMs is undermined by the fact that their output is frequently stated in very confident terms, even when its accuracy is questionable. Therefore, there is a need to signal the confidence of the language model to a user in order to reap the benefits of human-machine collaboration and mitigate potential harms. Verbalized uncertainty is the expression of confidence with linguistic means, an approach that integrates perfectly into language-based interfaces. Nevertheless, most recent research in natural language processing (NLP) overlooks the nuances surrounding human uncertainty communication and the data biases that influence machine uncertainty communication. We argue for anthropomimetic uncertainty, meaning that intuitive and trustworthy uncertainty communication requires a degree of linguistic authenticity and personalization to the user, which could be achieved by emulating human communication. We present a thorough overview over the research in human uncertainty communication, survey ongoing research, and perform additional analyses to demonstrate so-far overlooked biases in verbalized uncertainty. We conclude by pointing out unique factors in human-machine communication of uncertainty and deconstruct anthropomimetic uncertainty into future research directions for NLP.

Problem

• LLMs often express overconfidence despite questionable accuracy
• Current NLP research neglects human uncertainty communication nuances
• Anthropomimetic uncertainty needs linguistic authenticity for trust

Innovation

• Emulate human uncertainty communication linguistically
• Personalize uncertainty signals for user trust
• Address biases in machine uncertainty communication