Structure-Internalized Rule Language Model for Faithful Knowledge Graph Reasoning

2026-08-18

AI Summary

为解决大语言模型在知识图谱推理中证据感知偏差问题，提出了一种结构内化规则语言模型（SIRLM），通过生成结构规则来增强模型的推理逻辑忠实度。

Abstract

Knowledge Graph Reasoning (KGR) aims to discover latent facts by leveraging the structural evidence available in KGs, posing a challenge to the structural semantic understanding capability of KGR models. Recent studies have demonstrated that Large Language Models (LLMs) can achieve remarkable progress on KGR tasks via flexible in-context learning. However, the inherent representation inconsistency between KG structural context and LLM parametric knowledge remains inadequately addressed. This limitation prevents LLMs from effectively perceiving reasoning evidence that aligns with KG constraints, which undermines both the effectiveness and faithfulness of reasoning. We refer to this problem as reasoning evidence perception drift of LLMs over KGs. To address this problem, we propose a Structure-Internalized Rule Language Model (SIRLM), which centers on structural rule generation to couple the parametric learning of structural knowledge with the faithfulness evaluation of reasoning logic, enabling LLMs to anchor tightly to KG-grounded evidence. Specifically, we first design a Structure-Internalized Rule Generator (SIRG), which incorporates an in-context learning block augmented with a structural relation memory to coordinate structural and parametric knowledge. Furthermore, we equip SIRG with a KG tokenizer based on structural invariance learning and a neuro-symbolic reasoner based on rule-constrained message propagation. These components provide SIRG with learnable structural representations and faithful rule-execution feedback, respectively. Our SIRLM can be seamlessly integrated into standard LLM training paradigms, such as SFT and GRPO. Extensive experiments against 17 state-of-the-art KGR methods on 36 datasets demonstrate the significant superiority of SIRLM.

Problem

• Knowledge Graph Reasoning
• Large Language Models
• representation inconsistency
• reasoning evidence perception drift
• faithfulness

Innovation

• Structure-Internalized Rule Language Model
• Reasoning Evidence Perception Drift
• Knowledge Graph Reasoning
• Structural Invariance Learning
• Neuro-Symbolic Reasoner