SLiNT: Structure-aware Language Model with Injection and Contrastive Training for Knowledge Graph Completion

2025-09-08

AI Summary

To address the weak structural awareness of large language models (LLMs) and their poor generalization in sparse and zero-shot settings for knowledge graph (KG) link prediction, this paper proposes a lightweight, modular framework that injects graph-structural signals into frozen LLMs. The method comprises three core innovations: (1) Structural Graph Neighborhood Enhancement (SGNE) to mitigate structural sparsity; (2) Dynamic Hard-example Contrastive Learning (DHCL) to strengthen semantic discriminability; and (3) Gradient-Decoupled Dual Injection (GDDI), enabling joint optimization of structural and semantic representations. Leveraging LoRA-based adaptation, the framework avoids full LLM fine-tuning. Extensive experiments on WN18RR and FB15k-237 demonstrate state-of-the-art or competitive performance, validating the effectiveness and generalizability of structure-aware training for scalable KG completion.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Structured Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Link prediction in knowledge graphs requires integrating structural information and semantic context to infer missing entities. While large language models offer strong generative reasoning capabilities, their limited exploitation of structural signals often results in structural sparsity and semantic ambiguity, especially under incomplete or zero-shot settings. To address these challenges, we propose SLiNT (Structure-aware Language model with Injection and coNtrastive Training), a modular framework that injects knowledge-graph-derived structural context into a frozen LLM backbone with lightweight LoRA-based adaptation for robust link prediction. Specifically, Structure-Guided Neighborhood Enhancement (SGNE) retrieves pseudo-neighbors to enrich sparse entities and mitigate missing context; Dynamic Hard Contrastive Learning (DHCL) introduces fine-grained supervision by interpolating hard positives and negatives to resolve entity-level ambiguity; and Gradient-Decoupled Dual Injection (GDDI) performs token-level structure-aware intervention while preserving the core LLM parameters. Experiments on WN18RR and FB15k-237 show that SLiNT achieves superior or competitive performance compared with both embedding-based and generation-based baselines, demonstrating the effectiveness of structure-aware representation learning for scalable knowledge graph completion.

Problem

• Enhancing structural context integration for link prediction
• Mitigating structural sparsity and semantic ambiguity issues
• Improving zero-shot and incomplete knowledge graph completion

Innovation

• Structure injection via lightweight LoRA adaptation
• Dynamic contrastive learning with hard examples
• Gradient-decoupled token-level structure intervention