Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Models

2026-02-02

AI Summary

This study investigates whether large language models (LLMs) exhibit belief-guided autonomous behavior and metacognitive monitoring capabilities, as indicators of consciousness-like properties. Grounded in the neuroscientific Higher-Order Thought theory (HOT-3), the work introduces a novel quantitative measure of belief dominance, integrating latent-space belief representation modeling, causal intervention experiments, and a metacognitive self-reporting mechanism to construct an evaluation framework for belief–behavior alignment. Empirical results demonstrate that external interventions can systematically modulate internal belief formation within LLMs, that these beliefs causally drive action selection, and that the models can effectively monitor and report their own belief states. These findings provide new evidence and a methodological paradigm for belief-guided autonomy and metacognitive capacity in large language models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Rapid advancements in large language models (LLMs) have sparked the question whether these models possess some form of consciousness. To tackle this challenge, Butlin et al. (2023) introduced a list of indicators for consciousness in artificial systems based on neuroscientific theories. In this work, we evaluate a key indicator from this list, called HOT-3, which tests for agency guided by a general belief-formation and action selection system that updates beliefs based on meta-cognitive monitoring. We view beliefs as representations in the model's latent space that emerge in response to a given input, and introduce a metric to quantify their dominance during generation. Analyzing the dynamics between competing beliefs across models and tasks reveals three key findings: (1) external manipulations systematically modulate internal belief formation, (2) belief formation causally drives the model's action selection, and (3) models can monitor and report their own belief states. Together, these results provide empirical support for the existence of belief-guided agency and meta-cognitive monitoring in LLMs. More broadly, our work lays methodological groundwork for investigating the emergence of agency, beliefs, and meta-cognition in LLMs.

Problem

• belief-guided agency
• meta-cognitive monitoring
• large language models
• consciousness indicators
• HOT-3

Innovation

• belief-guided agency
• meta-cognitive monitoring
• latent belief representation
• causal belief-action dynamics
• consciousness indicators