RAG-GFM: Overcoming In-Memory Bottlenecks in Graph Foundation Models via Retrieval-Augmented Generation

2026-01-21

AI Summary

This work addresses the limitations of graph foundation models, which suffer from memory bottlenecks that restrict their semantic capacity, incur significant information loss during compression, and hinder interpretability. To overcome these challenges, we propose RAG-GFM, the first framework to integrate retrieval-augmented generation (RAG) into graph foundation models. RAG-GFM constructs a dual-modality retrieval corpus—comprising prefix-structured textual representations and centrality-based motifs—and introduces a dual-view contrastive alignment mechanism along with a context-enhanced adaptation strategy to enable externalized knowledge storage and efficient model adaptation. Evaluated on five benchmark datasets, RAG-GFM consistently outperforms 13 state-of-the-art baselines, achieving superior performance and efficiency in both cross-domain node classification and graph-level classification tasks.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Knowledge Representation and Reasoning: Other Foundations of Knowledge Representation & Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Graph Foundation Models (GFMs) have emerged as a frontier in graph learning, which are expected to deliver transferable representations across diverse tasks. However, GFMs remain constrained by in-memory bottlenecks: they attempt to encode knowledge into model parameters, which limits semantic capacity, introduces heavy lossy compression with conflicts, and entangles graph representation with the knowledge in ways that hinder efficient adaptation, undermining scalability and interpretability. In this work,we propose RAG-GFM, a Retrieval-Augmented Generation aided Graph Foundation Model that offloads knowledge from parameters and complements parameterized learning. To externalize graph knowledge, we build a dual-modal unified retrieval module, where a semantic store from prefix-structured text and a structural store from centrality-based motif. To preserve heterogeneous information, we design a dual-view alignment objective that contrasts both modalities to capture both content and relational patterns. To enable efficient downstream adaptation, we perform in-context augmentation to enrich supporting instances with retrieved texts and motifs as contextual evidence. Extensive experiments on five benchmark graph datasets demonstrate that RAG-GFM consistently outperforms 13 state-of-the-art baselines in both cross-domain node and graph classification, achieving superior effectiveness and efficiency.

Problem

• Graph Foundation Models
• in-memory bottlenecks
• knowledge entanglement
• scalability
• interpretability

Innovation

• Retrieval-Augmented Generation
• Graph Foundation Models
• Dual-Modal Retrieval
• In-Context Augmentation
• Motif-Based Structural Store