Metacognition and Uncertainty Communication in Humans and Large Language Models

2025-04-18

AI Summary

This study investigates whether, how, and to what extent large language models (LLMs) exhibit human-like metacognitive capabilities—specifically, monitoring, evaluating, and expressing uncertainty about their own knowledge states. Methodologically, we introduce the first **human-machine comparable metacognitive analysis framework**, integrating cognitive psychology experimental paradigms, LLM response behavior modeling, uncertainty quantification, and cross-subject benchmarking. Our results reveal that LLMs display non-introspective “pseudo-metacognition,” characterized by systematic miscalibration and structural deficits in uncertainty estimation. Building on these findings, we propose the concept of **calibrated metacognition**, defining it as a critical pathway toward trustworthy AI self-monitoring and adaptive learning. This work establishes foundational theory, rigorous evaluation criteria, and concrete directions for improving metacognitive reliability in AI systems. (136 words)

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Metacognition, the capacity to monitor and evaluate one's own knowledge and performance, is foundational to human decision-making, learning, and communication. As large language models (LLMs) become increasingly embedded in high-stakes decision contexts, it is critical to assess whether, how, and to what extent they exhibit metacognitive abilities. Here, we provide an overview of current knowledge of LLMs' metacognitive capacities, how they might be studied, and how they relate to our knowledge of metacognition in humans. We show that while humans and LLMs can sometimes appear quite aligned in their metacognitive capacities and behaviors, it is clear many differences remain. Attending to these differences is crucial not only for enhancing human-AI collaboration, but also for promoting the development of more capable and trustworthy artificial systems. Finally, we discuss how endowing future LLMs with more sensitive and more calibrated metacognition may also help them develop new capacities such as more efficient learning, self-direction, and curiosity.

Problem

• Assessing metacognitive abilities in large language models
• Comparing human and LLM metacognitive capacities and behaviors
• Enhancing LLMs with better metacognition for new capacities

Innovation

• Assessing metacognitive abilities in large language models
• Comparing human and LLM metacognitive capacities and behaviors
• Enhancing LLMs with calibrated metacognition for new capacities