Human-AI Collaborative Uncertainty Quantification

2025-10-27

AI Summary

AI systems are increasingly deployed in high-stakes decision-making but lack domain knowledge not captured in training data, long-horizon contextual reasoning, and physical intuition—limiting their robustness under uncertainty. Method: We propose a human-AI collaborative uncertainty quantification framework: (i) a dual-threshold structural theory to construct cooperative prediction sets that preserve human correctness, prevent counterfactual harm, and enable complementarity; (ii) an online calibration algorithm adaptive to distributional shift, mitigating behavioral evolution induced by human-AI interaction; and (iii) an extension of conformal prediction yielding finite-sample theoretical guarantees without distributional assumptions. Contribution/Results: Experiments on image classification, regression, and clinical text-based decision tasks demonstrate that our method significantly improves coverage over standalone human or AI baselines while reducing prediction set size—achieving both reliability and efficiency in uncertain environments.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

AI predictive systems are increasingly embedded in decision making pipelines, shaping high stakes choices once made solely by humans. Yet robust decisions under uncertainty still rely on capabilities that current AI lacks: domain knowledge not captured by data, long horizon context, and reasoning grounded in the physical world. This gap has motivated growing efforts to design collaborative frameworks that combine the complementary strengths of humans and AI. This work advances this vision by identifying the fundamental principles of Human AI collaboration within uncertainty quantification, a key component of reliable decision making. We introduce Human AI Collaborative Uncertainty Quantification, a framework that formalizes how an AI model can refine a human expert's proposed prediction set with two goals: avoiding counterfactual harm, ensuring the AI does not degrade correct human judgments, and complementarity, enabling recovery of correct outcomes the human missed. At the population level, we show that the optimal collaborative prediction set follows an intuitive two threshold structure over a single score function, extending a classical result in conformal prediction. Building on this insight, we develop practical offline and online calibration algorithms with provable distribution free finite sample guarantees. The online method adapts to distribution shifts, including human behavior evolving through interaction with AI, a phenomenon we call Human to AI Adaptation. Experiments across image classification, regression, and text based medical decision making show that collaborative prediction sets consistently outperform either agent alone, achieving higher coverage and smaller set sizes across various conditions.

Problem

• Bridging AI limitations in domain knowledge and uncertainty quantification
• Developing human-AI collaboration frameworks for reliable decision making
• Creating calibration algorithms with provable guarantees under distribution shifts

Innovation

• Human-AI framework refines expert prediction sets
• Two-threshold structure extends conformal prediction methods
• Online calibration adapts to human behavior shifts