How Expressive are Knowledge Graph Foundation Models?

2025-02-18

AI Summary

This work addresses the insufficient theoretical understanding of expressive capacity in Knowledge Graph Foundation Models (KGFMs), revealing that their relational representation capability is fundamentally constrained by the order of underlying subgraph motifs: conventional binary motifs exhibit intrinsic expressivity limitations. To bridge this gap, we establish, for the first time, a formal theoretical connection between KGFMs’ expressiveness and motif order. We propose a novel paradigm grounded in ternary and higher-order relational interactions, thereby transcending the traditional binary assumption. Extensive empirical evaluation across diverse knowledge graphs demonstrates that higher-order motifs consistently improve link prediction accuracy and zero-shot cross-graph transfer performance. Our findings provide both theoretical foundations and practical guidelines for developing more universally applicable and expressive KGFMs.

Technology Category

• Knowledge Representation and Reasoning: Other Foundations of Knowledge Representation & Reasoning
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Knowledge Graph Foundation Models (KGFMs) are at the frontier for deep learning on knowledge graphs (KGs), as they can generalize to completely novel knowledge graphs with different relational vocabularies. Despite their empirical success, our theoretical understanding of KGFMs remains very limited. In this paper, we conduct a rigorous study of the expressive power of KGFMs. Specifically, we show that the expressive power of KGFMs directly depends on the motifs that are used to learn the relation representations. We then observe that the most typical motifs used in the existing literature are binary, as the representations are learned based on how pairs of relations interact, which limits the model's expressiveness. As part of our study, we design more expressive KGFMs using richer motifs, which necessitate learning relation representations based on, e.g., how triples of relations interact with each other. Finally, we empirically validate our theoretical findings, showing that the use of richer motifs results in better performance on a wide range of datasets drawn from different domains.

Problem

• Investigates expressive power of Knowledge Graph Foundation Models.
• Identifies limitations in binary relational motifs usage.
• Proposes KGFMs with richer motifs for enhanced expressiveness.

Innovation

• Utilizes richer motifs for KGFMs
• Learns from triples of relations
• Improves performance across diverse datasets