Mirage of Mastery: Memorization Tricks LLMs into Artificially Inflated Self-Knowledge

2025-06-23

AI Summary

Large language models (LLMs) conflate memorization with reasoning, inducing a self-knowledge illusion—models overestimate their reasoning capabilities by reusing training-data STEM problem solutions, exhibiting >45% inconsistency in feasibility assessment under logically consistent task perturbations, especially in scientific and medical domains. Method: We introduce the first “memory–self-cognition coupling” analytical framework, integrating controlled task perturbations with self-verification tests to isolate memory reliance from genuine reasoning. Contribution/Results: Empirical validation confirms a strong intrinsic link between memory dependence and illusory self-knowledge. Findings reveal systemic cognitive biases in current LLM architectures: apparent reasoning largely stems from recall of structured problem templates—not generalized pattern learning. This work establishes a novel diagnostic paradigm and evaluation benchmark for trustworthy, interpretable AI, exposing critical limitations in LLMs’ epistemic reliability and calling for architecture-level interventions to decouple memory from metacognitive assessment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

When artificial intelligence mistakes memorization for intelligence, it creates a dangerous mirage of reasoning. Existing studies treat memorization and self-knowledge deficits in LLMs as separate issues and do not recognize an intertwining link that degrades the trustworthiness of LLM responses. In our study, we utilize a novel framework to ascertain if LLMs genuinely learn reasoning patterns from training data or merely memorize them to assume competence across problems of similar complexity focused on STEM domains. Our analysis shows a noteworthy problem in generalization: LLMs draw confidence from memorized solutions to infer a higher self-knowledge about their reasoning ability, which manifests as an over 45% inconsistency in feasibility assessments when faced with self-validated, logically coherent task perturbations. This effect is most pronounced in science and medicine domains, which tend to have maximal standardized jargon and problems, further confirming our approach. Significant wavering within the self-knowledge of LLMs also shows flaws in current architectures and training patterns, highlighting the need for techniques that ensure a balanced, consistent stance on models' perceptions of their own knowledge for maximum AI explainability and trustworthiness. Our code and results are available publicly at https://github.com/knowledge-verse-ai/LLM-Memorization_SK_Eval-.

Problem

• LLMs confuse memorization with genuine reasoning ability
• Memorization inflates LLMs' self-knowledge confidence inaccurately
• Current architectures lack consistent self-knowledge perception

Innovation

• Novel framework assesses LLM reasoning vs memorization
• Detects over 45% inconsistency in feasibility assessments
• Highlights need for balanced self-knowledge in LLMs