Do Chains-of-Thoughts of Large Language Models Suffer from Hallucinations, Cognitive Biases, or Phobias in Bayesian Reasoning?

2025-03-19

AI Summary

This study investigates whether large language models (LLMs) exhibit hallucinations, cognitive biases, and a structural avoidance—termed “strategic aversion”—of ecologically valid Bayesian reasoning strategies, such as natural frequency representations and embodied heuristics, when employing chain-of-thought (CoT) reasoning. Method: We construct a novel Bayesian reasoning benchmark and employ comparative analysis, strategy-induction prompting, and CoT reasoning evaluation to systematically probe LLM behavior. Contribution/Results: We provide the first systematic evidence that LLMs display strong symbolic bias and strategy inconsistency: although capable of computing with natural frequencies, they consistently avoid deploying them. Building on this finding, we propose a bias-informed prompting framework. Empirical evaluation demonstrates that this framework partially mitigates strategic aversion and enhances reasoning robustness. Our work advances AI-augmented mathematics education by offering a human-cognition-grounded, interpretable optimization pathway for LLM-based reasoning systems.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Learning to reason and carefully explain arguments is central to students' cognitive, mathematical, and computational thinking development. This is particularly challenging in problems under uncertainty and in Bayesian reasoning. With the new generation of large language models (LLMs) capable of reasoning using Chain-of-Thought (CoT), there is an excellent opportunity to learn with them as they explain their reasoning through a dialogue with their artificial internal voice. It is an engaging and excellent opportunity to learn Bayesian reasoning. Furthermore, given that different LLMs sometimes arrive at opposite solutions, CoT generates opportunities for deep learning by detailed comparisons of reasonings. However, unlike humans, we found that they do not autonomously explain using ecologically valid strategies like natural frequencies, whole objects, and embodied heuristics. This is unfortunate, as these strategies help humans avoid critical mistakes and have proven pedagogical value in Bayesian reasoning. In order to overcome these biases and aid understanding and learning, we included prompts that induce LLMs to use these strategies. We found that LLMs with CoT incorporate them but not consistently. They show persistent biases towards symbolic reasoning and avoidance or phobia of ecologically valid strategies.

Problem

• Assessing LLMs' CoT for hallucinations, biases, phobias in Bayesian reasoning.
• Exploring LLMs' inconsistent use of ecologically valid reasoning strategies.
• Enhancing LLMs' Bayesian reasoning with prompts for better learning outcomes.

Innovation

• Utilizes Chain-of-Thought for Bayesian reasoning
• Introduces prompts for ecologically valid strategies
• Analyzes biases in symbolic reasoning avoidance