Turning Tabular Foundation Models into Graph Foundation Models

2025-08-28

AI Summary

Graph Foundation Models (GFMs) struggle to uniformly model heterogeneous node features—particularly non-textual attributes—due to architectural biases toward specific modalities. Method: This paper proposes G2T-FM, the first framework to integrate the tabular foundation model TabPFNv2 into graph learning. It constructs node representations via neighborhood feature aggregation and structural embedding, and incorporates in-context learning—eliminating the need for graph neural networks (GNNs) while enabling end-to-end node prediction. Contribution/Results: G2T-FM establishes a simple, general, and feature-agnostic GFM paradigm natively compatible with arbitrary node attribute types. Experiments show that, under full in-context learning, G2T-FM outperforms existing open GFMs and matches the performance of GNNs trained from scratch; with lightweight fine-tuning, it significantly surpasses strong GNN baselines. These results demonstrate the substantial potential of tabular foundation models for graph representation learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

While foundation models have revolutionized such fields as natural language processing and computer vision, their application and potential within graph machine learning remain largely unexplored. One of the key challenges in designing graph foundation models (GFMs) is handling diverse node features that can vary across different graph datasets. Although many works on GFMs have been focused exclusively on text-attributed graphs, the problem of handling arbitrary features of other types in GFMs has not been fully addressed. However, this problem is not unique to the graph domain, as it also arises in the field of machine learning for tabular data. In this work, motivated by the recent success of tabular foundation models like TabPFNv2, we propose G2T-FM, a simple graph foundation model that employs TabPFNv2 as a backbone. Specifically, G2T-FM augments the original node features with neighborhood feature aggregation, adds structural embeddings, and then applies TabPFNv2 to the constructed node representations. Even in a fully in-context regime, our model achieves strong results, significantly outperforming publicly available GFMs and performing on par with well-tuned GNNs trained from scratch. Moreover, after finetuning, G2T-FM surpasses well-tuned GNN baselines, highlighting the potential of the proposed approach. More broadly, our paper reveals a previously overlooked direction of utilizing tabular foundation models for graph machine learning tasks.

Problem

• Handling diverse node features in graph foundation models
• Addressing arbitrary feature types beyond text-attributed graphs
• Applying tabular foundation models to graph machine learning tasks

Innovation

• Uses TabPFNv2 as backbone for graph foundation model
• Augments node features with neighborhood aggregation
• Adds structural embeddings to constructed node representations