Mitigating Heterogeneity among Factor Tensors via Lie Group Manifolds for Tensor Decomposition Based Temporal Knowledge Graph Embedding

2024-04-14

arXiv.org

AI Summary

Tensor decomposition methods for temporal knowledge graph embedding (TKGE) suffer from factor tensor heterogeneity, severely degrading tensor fusion quality and link prediction performance. Method: This paper proposes, for the first time, mapping all factor tensors onto a Lie group manifold to achieve geometrically consistent (homogeneous) representations under differential-geometric constraints. The mapping introduces no additional trainable parameters and is theoretically grounded: homogeneous tensors facilitate better objective function approximation and more efficient fusion. Contribution/Results: The method is plug-and-play and compatible with mainstream tensor decomposition TKGE models. Extensive experiments on multiple benchmark TKGE datasets demonstrate significant improvements in link prediction accuracy, effectively alleviating the factor heterogeneity bottleneck. This work establishes a novel geometric-prior-driven paradigm for TKGE modeling.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Matrix & Tensor Methods
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent studies have highlighted the effectiveness of tensor decomposition methods in the Temporal Knowledge Graphs Embedding (TKGE) task. However, we found that inherent heterogeneity among factor tensors in tensor decomposition significantly hinders the tensor fusion process and further limits the performance of link prediction. To overcome this limitation, we introduce a novel method that maps factor tensors onto a unified smooth Lie group manifold to make the distribution of factor tensors approximating homogeneous in tensor decomposition. We provide the theoretical proof of our motivation that homogeneous tensors are more effective than heterogeneous tensors in tensor fusion and approximating the target for tensor decomposition based TKGE methods. The proposed method can be directly integrated into existing tensor decomposition based TKGE methods without introducing extra parameters. Extensive experiments demonstrate the effectiveness of our method in mitigating the heterogeneity and in enhancing the tensor decomposition based TKGE models.

Problem

• Address heterogeneity in factor tensors
• Enhance tensor decomposition for TKGE
• Map tensors to Lie group manifold

Innovation

• Lie group manifold mapping
• Homogeneous tensor distribution
• Enhanced TKGE model performance