Beyond Completion: A Foundation Model for General Knowledge Graph Reasoning

2025-05-28

AI Summary

Existing knowledge graph (KG) foundation models primarily target in-graph tasks (e.g., link prediction), exhibiting limited generalization to out-of-graph tasks (e.g., question answering). This work introduces MERRY, the first general-purpose KG reasoning foundation model unifying in-graph and out-of-graph inference. Methodologically, MERRY features: (1) a multi-view conditional message-passing encoder for structure-aware graph representation learning; (2) a dynamic residual text-fusion module that explicitly aligns textual and graph modalities; and (3) an adaptable edge-scoring mechanism enabling end-to-end differentiable reasoning. Trained under the pretraining-finetuning paradigm, MERRY achieves state-of-the-art performance across 28 benchmark datasets. It delivers substantial gains in both knowledge graph completion (KGC) and KG-based question answering (KGQA), while demonstrating markedly improved cross-task generalization capability.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Other Foundations of Knowledge Representation & Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

In natural language processing (NLP) and computer vision (CV), the successful application of foundation models across diverse tasks has demonstrated their remarkable potential. However, despite the rich structural and textual information embedded in knowledge graphs (KGs), existing research of foundation model for KG has primarily focused on their structural aspects, with most efforts restricted to in-KG tasks (e.g., knowledge graph completion, KGC). This limitation has hindered progress in addressing more challenging out-of-KG tasks. In this paper, we introduce MERRY, a foundation model for general knowledge graph reasoning, and investigate its performance across two task categories: in-KG reasoning tasks (e.g., KGC) and out-of-KG tasks (e.g., KG question answering, KGQA). We not only utilize the structural information, but also the textual information in KGs. Specifically, we propose a multi-perspective Conditional Message Passing (CMP) encoding architecture to bridge the gap between textual and structural modalities, enabling their seamless integration. Additionally, we introduce a dynamic residual fusion module to selectively retain relevant textual information and a flexible edge scoring mechanism to adapt to diverse downstream tasks. Comprehensive evaluations on 28 datasets demonstrate that MERRY outperforms existing baselines in most scenarios, showcasing strong reasoning capabilities within KGs and excellent generalization to out-of-KG tasks such as KGQA.

Problem

• Bridging textual and structural modalities in KGs
• Enhancing reasoning for in-KG and out-of-KG tasks
• Improving generalization in knowledge graph question answering

Innovation

• Multi-perspective Conditional Message Passing encoding
• Dynamic residual fusion module
• Flexible edge scoring mechanism