Rethinking Regularization Methods for Knowledge Graph Completion

2025-05-29

AI Summary

Existing knowledge graph completion (KGC) regularization methods predominantly rely on shallow constraints—such as L2 norm or DropEdge—which fail to exploit intrinsic structural regularities within embedding spaces, thereby limiting model capacity. To address this, we propose SPR (Selective Penalty via Rank), a novel regularization paradigm that introduces a dynamic, rank-aware sparsity mechanism. SPR first performs spectral analysis of entity and relation embeddings to identify dominant singular components; it then applies importance-weighted penalties proportional to component significance, effectively suppressing noise while enhancing generalization. Evaluated across standard benchmarks—including FB15k-237 and WN18RR—and mainstream models (TransE, RotatE, ComplEx), SPR consistently outperforms conventional baselines (e.g., L2, N3), yielding average MRR improvements of 2.1–4.7%. This marks the first instance of a regularization-driven, substantive performance breakthrough in KGC.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Knowledge graph completion (KGC) has attracted considerable attention in recent years because it is critical to improving the quality of knowledge graphs. Researchers have continuously explored various models. However, most previous efforts have neglected to take advantage of regularization from a deeper perspective and therefore have not been used to their full potential. This paper rethinks the application of regularization methods in KGC. Through extensive empirical studies on various KGC models, we find that carefully designed regularization not only alleviates overfitting and reduces variance but also enables these models to break through the upper bounds of their original performance. Furthermore, we introduce a novel sparse-regularization method that embeds the concept of rank-based selective sparsity into the KGC regularizer. The core idea is to selectively penalize those components with significant features in the embedding vector, thus effectively ignoring many components that contribute little and may only represent noise. Various comparative experiments on multiple datasets and multiple models show that the SPR regularization method is better than other regularization methods and can enable the KGC model to further break through the performance margin.

Problem

• Improving knowledge graph completion via advanced regularization
• Addressing overfitting and variance in KGC models with regularization
• Introducing sparse-regularization for selective feature penalization in KGC

Innovation

• Rethinks regularization methods for KGC
• Introduces rank-based sparse-regularization method
• Selectively penalizes significant embedding components