Enhancing Large Language Model for Knowledge Graph Completion via Structure-Aware Alignment-Tuning

2025-09-01

AI Summary

To address the misalignment between natural language and graph-structured representations, and the redundant design of task-specific instructions in LLM-enhanced knowledge graph completion (KGC), this paper proposes a structure-aware alignment tuning framework. Methodologically, it achieves cross-modal representation unification via hierarchical knowledge alignment and supports structure-aware reasoning under generic instructions through structured unified graph instruction tuning—integrating multi-task contrastive learning, graph embedding alignment, and a lightweight knowledge adapter. Evaluated on four benchmark datasets, the framework improves link prediction performance by 8.7%–29.8%, demonstrating strong effectiveness and generalization. Its core contributions are twofold: (1) the first approach to jointly align natural language semantics with the geometric properties of graph structures; and (2) establishing a generic instruction paradigm for KGC, significantly reducing multi-task adaptation overhead.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Structured Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

Knowledge graph completion (KGC) aims to infer new knowledge and make predictions from knowledge graphs. Recently, large language models (LLMs) have exhibited remarkable reasoning capabilities. LLM-enhanced KGC methods primarily focus on designing task-specific instructions, achieving promising advancements. However, there are still two critical challenges. First, existing methods often ignore the inconsistent representation spaces between natural language and graph structures. Second, most approaches design separate instructions for different KGC tasks, leading to duplicate works and time-consuming processes. To address these challenges, we propose SAT, a novel framework that enhances LLMs for KGC via structure-aware alignment-tuning. Specifically, we first introduce hierarchical knowledge alignment to align graph embeddings with the natural language space through multi-task contrastive learning. Then, we propose structural instruction tuning to guide LLMs in performing structure-aware reasoning over KGs, using a unified graph instruction combined with a lightweight knowledge adapter. Experimental results on two KGC tasks across four benchmark datasets demonstrate that SAT significantly outperforms state-of-the-art methods, especially in the link prediction task with improvements ranging from 8.7% to 29.8%.

Problem

• Aligns graph embeddings with natural language space
• Unifies instructions for different knowledge graph tasks
• Enables structure-aware reasoning in language models

Innovation

• Structure-aware alignment-tuning for graph embeddings
• Multi-task contrastive learning for language alignment
• Unified graph instruction with lightweight knowledge adapter