Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement

2025-11-03

AI Summary

This study investigates the dynamic interplay between parametric knowledge (PK) and contextual knowledge (CK) during natural language explanation (NLE) generation by large language models (LLMs). Addressing the limitation of conventional rank-1 modeling—which fails to capture the complexity of multi-step knowledge integration—we propose the first rank-2 subspace disentanglement framework for analyzing multi-step PK–CK interaction. Our method combines directional attribution with sequence-level tracing to enable fine-grained, stepwise quantification of PK and CK contributions on open-weight instruction-tuned LLMs. Experiments across four QA datasets and three mainstream LLMs reveal that hallucinated explanations exhibit strong PK bias, faithful explanations reflect balanced PK–CK integration, and chain-of-thought prompting enhances CK reliance. This work provides the first empirical characterization of the dynamic knowledge equilibrium underlying NLE generation, establishing a new paradigm for controllable knowledge injection and hallucination mitigation.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Learning Preferences or Rankings
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions, drawing on both external Context Knowledge (CK) and Parametric Knowledge (PK) stored in model weights. Understanding their interaction is key to assessing the grounding of NLEs, yet it remains underexplored. Prior work has largely examined only single-step generation, typically the final answer, and has modelled PK and CK interaction only as a binary choice in a rank-1 subspace. This overlooks richer forms of interaction, such as complementary or supportive knowledge. We propose a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and use it for the first multi-step analysis of knowledge interactions across longer NLE sequences. Experiments on four QA datasets and three open-weight instruction-tuned LLMs show that diverse knowledge interactions are poorly represented in a rank-1 subspace but are effectively captured in our rank-2 formulation. Our multi-step analysis reveals that hallucinated NLEs align strongly with the PK direction, context-faithful ones balance PK and CK, and Chain-of-Thought prompting for NLEs shifts generated NLEs toward CK by reducing PK reliance. This work provides the first framework for systematic studies of multi-step knowledge interactions in LLMs through a richer rank-2 subspace disentanglement. Code and data: https://github.com/copenlu/pk-ck-knowledge-disentanglement.

Problem

• Analyzes multi-step knowledge interactions in LLM explanations
• Disentangles parametric and contextual knowledge via rank-2 subspace
• Reveals knowledge interaction patterns across explanation generation steps

Innovation

• Rank-2 subspace disentangles PK and CK contributions
• Multi-step analysis of knowledge interactions in NLEs
• Framework captures diverse interactions beyond binary choice