Beyond Quantification: Navigating Uncertainty in Professional AI Systems

2025-09-03

AI Summary

Conventional probabilistic uncertainty quantification in AI systems (e.g., confidence scores) fails to capture inherently non-quantifiable judgmental uncertainty in high-stakes professional domains—such as domestic violence risk assessment, cultural sensitivity evaluation, or conceptual understanding recognition—where uncertainty is epistemically and contextually grounded rather than statistically estimable. Method: We propose a non-quantitative, meaning-oriented uncertainty representation paradigm that reframes uncertainty articulation as a collaborative meaning-making process within professional communities. Integrating human-computer interaction, practice theory, and participatory design, we develop a co-evolving refinement mechanism wherein domain experts iteratively define and optimize uncertainty expression forms. Contribution/Results: We establish the first theoretical framework for non-quantitative uncertainty tailored to professional practice and empirically validate its efficacy across multiple domains, demonstrating significant improvements in expert–AI collaborative decision quality.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

The growing integration of large language models across professional domains transforms how experts make critical decisions in healthcare, education, and law. While significant research effort focuses on getting these systems to communicate their outputs with probabilistic measures of reliability, many consequential forms of uncertainty in professional contexts resist such quantification. A physician pondering the appropriateness of documenting possible domestic abuse, a teacher assessing cultural sensitivity, or a mathematician distinguishing procedural from conceptual understanding face forms of uncertainty that cannot be reduced to percentages. This paper argues for moving beyond simple quantification toward richer expressions of uncertainty essential for beneficial AI integration. We propose participatory refinement processes through which professional communities collectively shape how different forms of uncertainty are communicated. Our approach acknowledges that uncertainty expression is a form of professional sense-making that requires collective development rather than algorithmic optimization.

Problem

• Addressing non-quantifiable uncertainties in professional AI systems
• Developing richer uncertainty expressions beyond probabilistic measures
• Creating participatory processes for professional uncertainty communication

Innovation

• Participatory refinement processes for uncertainty communication
• Collective professional development over algorithmic optimization
• Richer expressions beyond probabilistic quantification methods