Can Graph Foundation Models Generalize Over Architecture?

2026-03-24

AI Summary

This work addresses the limitations of existing graph foundation models, whose fixed architectures struggle to accommodate diverse task-specific requirements for message passing, thereby constraining zero-shot generalization and robustness. To overcome this, we introduce architecture adaptability into graph foundation models for the first time, proposing a method that dynamically blends task-aware linear graph operators during inference to construct a task-optimal GNN architecture without retraining. Theoretical analysis and controlled experiments demonstrate the efficacy of our approach, which significantly outperforms current domain-agnostic graph foundation models on both synthetic tasks and multiple real-world benchmarks, markedly enhancing cross-task generalization and robustness.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Graph foundation models (GFMs) have recently attracted interest due to the promise of graph neural network (GNN) architectures that generalize zero-shot across graphs of arbitrary scales, feature dimensions, and domains. While existing work has demonstrated this ability empirically across diverse real-world benchmarks, these tasks share a crucial hidden limitation: they admit a narrow set of effective GNN architectures. In particular, current domain-agnostic GFMs rely on fixed architectural backbones, implicitly assuming that a single message-passing regime suffices across tasks. In this paper, we argue that architecture adaptivity is a necessary requirement for true GFMs. We show that existing approaches are non-robust to task-dependent architectural attributes and, as a case study, use range as a minimal and measurable axis along which this limitation becomes explicit. With theoretical analysis and controlled synthetic experiments, we demonstrate that fixed-backbone GFMs provably under-reach on tasks whose architectural requirements differ from those seen at training time. To address this issue, we introduce a framework that adapts effective GNN architecture at inference time by discovering and mixing task-specific linear graph operators, enabling zero-shot generalization across tasks with heterogeneous architectural requirements, without retraining. We validate our approach on arbitrary-range synthetic tasks and a suite of real-world benchmarks, demonstrating improved performance and robustness over existing domain-agnostic GFMs.

Problem

• Graph Foundation Models
• Architecture Generalization
• Zero-shot Generalization
• GNN Architecture
• Task Adaptivity

Innovation

• Graph Foundation Models
• Architecture Adaptivity
• Zero-shot Generalization
• Linear Graph Operators
• Message-passing Regime