Non-Homophilic Graph Pre-Training and Prompt Learning

2024-08-22

arXiv.org

AI Summary

Existing graph neural network (GNN) pre-training and prompt learning methods overlook the dynamic evolution of node similarity/dissimilarity, particularly under non-homophilous graph structures—where homophilous and heterophilous patterns coexist—leading to suboptimal performance. Method: We propose ProNoG, the first pre-training and prompt learning framework tailored for non-homophilous graphs. It systematically analyzes how non-homophily affects pre-training objectives, introduces a conditional node-aware prompting mechanism to capture node-level heterophilous patterns, and adopts a dual-path design integrating theoretically grounded interpretability analysis with data-driven optimization. Contribution/Results: Evaluated on 10 public benchmarks, ProNoG consistently outperforms state-of-the-art methods, achieving an average accuracy gain of 7.2% on strongly heterophilous graphs. Ablation studies validate both the effectiveness and generalizability of node-level prompts, establishing ProNoG as a principled and empirically robust solution for non-homophilous graph representation learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Natural Language Processing: Prompt Engineering / Prompting
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Graphs are ubiquitous for modeling complex relationships between objects across various fields. Graph neural networks (GNNs) have become a mainstream technique for graph-based applications, but their performance heavily relies on abundant labeled data. To reduce labeling requirement, pre-training and prompt learning has become a popular alternative. However, most existing prompt methods do not differentiate homophilic and heterophilic characteristics of real-world graphs. In particular, many real-world graphs are non-homophilic, not strictly or uniformly homophilic with mixing homophilic and heterophilic patterns, exhibiting varying non-homophilic characteristics across graphs and nodes. In this paper, we propose ProNoG, a novel pre-training and prompt learning framework for such non-homophilic graphs. First, we analyze existing graph pre-training methods, providing theoretical insights into the choice of pre-training tasks. Second, recognizing that each node exhibits unique non-homophilic characteristics, we propose a conditional network to characterize the node-specific patterns in downstream tasks. Finally, we thoroughly evaluate and analyze ProNoG through extensive experiments on ten public datasets.

Problem

• Graph Neural Networks
• Pre-training
• Prompt Learning

Innovation

• Pretraining
• Graph Neural Networks
• Node-specific Dynamics