Equivariance Everywhere All At Once: A Recipe for Graph Foundation Models

2025-06-17

AI Summary

Graph machine learning models suffer from poor generalization and limited transferability across diverse graph structures and node features. Method: This paper introduces the first unified framework enforcing three fundamental symmetries—label permutation equivariance, feature permutation invariance, and local node equivariance—grounded in group representation theory, yielding a generic, theoretically grounded node-level prediction architecture. Contribution/Results: The proposed model is a universal approximator over multisets, combining theoretical soundness with structural interpretability. Evaluated on 29 real-world node classification benchmarks, it demonstrates strong zero-shot transfer capability across unseen graphs; moreover, its generalization performance improves consistently with increasing numbers of training graphs. These results significantly advance graph foundation models toward task-agnostic and data-agnostic universal paradigms.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Graph machine learning architectures are typically tailored to specific tasks on specific datasets, which hinders their broader applicability. This has led to a new quest in graph machine learning: how to build graph foundation models capable of generalizing across arbitrary graphs and features? In this work, we present a recipe for designing graph foundation models for node-level tasks from first principles. The key ingredient underpinning our study is a systematic investigation of the symmetries that a graph foundation model must respect. In a nutshell, we argue that label permutation-equivariance alongside feature permutation-invariance are necessary in addition to the common node permutation-equivariance on each local neighborhood of the graph. To this end, we first characterize the space of linear transformations that are equivariant to permutations of nodes and labels, and invariant to permutations of features. We then prove that the resulting network is a universal approximator on multisets that respect the aforementioned symmetries. Our recipe uses such layers on the multiset of features induced by the local neighborhood of the graph to obtain a class of graph foundation models for node property prediction. We validate our approach through extensive experiments on 29 real-world node classification datasets, demonstrating both strong zero-shot empirical performance and consistent improvement as the number of training graphs increases.

Problem

• Building graph foundation models for generalizing across arbitrary graphs
• Investigating symmetries for graph models: node and label permutation-equivariance
• Validating universal approximator for node property prediction tasks

Innovation

• Systematic investigation of graph symmetries
• Linear transformations respecting node-label-feature permutations
• Universal approximator for symmetric multiset networks