A Fair Evaluation of Graph Foundation Models for Node Property Prediction

2026-06-23

AI Summary

This study addresses the lack of a unified and fair evaluation protocol for graph foundation models (GFMs) in node attribute prediction tasks, which has hindered reliable performance comparisons with traditional graph neural networks (GNNs). To remedy this, the authors establish the first standardized evaluation framework that harmonizes datasets, training protocols, and evaluation metrics. Within this framework, they systematically reproduce and benchmark nine recent GFMs against carefully tuned GNN baselines, including GCN and GAT. The results demonstrate that, with the exception of the most recent GFM based on the Prior-data Fitted Networks paradigm, none of the evaluated GFMs consistently outperform optimized GNNs—while the top-performing GFMs incur substantially higher inference costs. This work thus provides a reliable benchmark for assessing GFM capabilities and exposes the practical limitations of current approaches.

Abstract

Due to the wide use of graph-structured data in different fields of industry and science, the development of Graph Foundation Models (GFMs) has recently attracted a lot of attention. While many different types of models are called GFMs, particular interest has been paid to GFMs designed for node property prediction tasks, which is one of the most popular settings in Graph ML with lots of real-world applications from fraud detection in financial and social networks to recommendation systems for e-commerce and user-generated content platforms. While a number of GFMs for this task have been recently proposed, the field has not converged to a unified evaluation setting, and different works evaluate their models in widely different ways, preventing reliable comparison of GFMs with each other and with other types of models. In this work, we conduct a fair and rigorous reevaluation of 9 recent GFMs for node property prediction, comparing them to strong Graph Neural Network (GNN) baselines. We find that, among these GFMs, only the most recent ones based on the Prior-data Fitted Networks paradigm outperform well-tuned GNNs in predictive performance, although at a higher inference cost.

Problem

• Graph Foundation Models
• Node Property Prediction
• Model Evaluation
• Graph Neural Networks
• Fair Comparison

Innovation

• Graph Foundation Models
• Node Property Prediction
• Fair Evaluation
• Graph Neural Networks
• Prior-data Fitted Networks