Beyond Accuracy: How AI Metacognitive Sensitivity improves AI-assisted Decision Making

2025-07-30

AI Summary

This study investigates how metacognitive sensitivity—the AI’s ability to discriminate between its own correct and incorrect predictions via confidence scores—affects human decision quality in AI-assisted decision-making, beyond mere predictive accuracy. We develop a theoretical model and conduct behavioral experiments to quantify the independent and interactive effects of accuracy and metacognitive sensitivity in human-AI collaborative decisions. Results show that high metacognitive sensitivity significantly improves human decision performance even when AI accuracy is low; moreover, jointly optimizing both dimensions yields superior outcomes compared to accuracy-only evaluation paradigms. This work challenges the conventional accuracy-centric framework for evaluating AI assistance and provides the first systematic empirical validation of metacognitive sensitivity as an independent contributor to decision efficacy. It establishes a dual-dimensional optimization principle—balancing accuracy and metacognitive sensitivity—for designing trustworthy AI systems. (149 words)

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Search and Optimization: Metareasoning and Metaheuristics
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

In settings where human decision-making relies on AI input, both the predictive accuracy of the AI system and the reliability of its confidence estimates influence decision quality. We highlight the role of AI metacognitive sensitivity -- its ability to assign confidence scores that accurately distinguish correct from incorrect predictions -- and introduce a theoretical framework for assessing the joint impact of AI's predictive accuracy and metacognitive sensitivity in hybrid decision-making settings. Our analysis identifies conditions under which an AI with lower predictive accuracy but higher metacognitive sensitivity can enhance the overall accuracy of human decision making. Finally, a behavioral experiment confirms that greater AI metacognitive sensitivity improves human decision performance. Together, these findings underscore the importance of evaluating AI assistance not only by accuracy but also by metacognitive sensitivity, and of optimizing both to achieve superior decision outcomes.

Problem

• AI metacognitive sensitivity enhances decision-making accuracy.
• Joint impact of AI accuracy and confidence reliability.
• Lower accuracy AI with high metacognition improves decisions.

Innovation

• AI metacognitive sensitivity enhances decision-making
• Theoretical framework assesses accuracy and sensitivity
• Behavioral experiment confirms sensitivity improves performance