🤖 AI Summary
This work addresses negative transfer in pre-trained heterogeneous graph neural networks (HGNNs) for semi-supervised node classification, caused by objective mismatch between pre-training and downstream tasks. We propose a novel “pre-training + prompting” paradigm. Our method introduces: (1) the first heterogeneous-graph-aware prompt function, jointly modeling virtual class prompts and heterogeneous feature prompts; and (2) a multi-view neighbor aggregation mechanism, enabling the first general adaptation of prompt tuning to HGNNs. Crucially, our approach requires no fine-tuning of model parameters—only lightweight prompt modules are optimized. Evaluated on three standard heterogeneous graph benchmarks, it consistently outperforms state-of-the-art HGNNs, achieving average accuracy gains of 3.2–5.8 percentage points. These results validate the effectiveness and generalizability of prompt-based alignment between pre-training objectives and downstream tasks.
📝 Abstract
Graphs have emerged as a natural choice to represent and analyze the intricate patterns and rich information of the Web, enabling applications such as online page classification and social recommendation. The prevailing ''pre-train, fine-tune'' paradigm has been widely adopted in graph machine learning tasks, particularly in scenarios with limited labeled nodes. However, this approach often exhibits a misalignment between the training objectives of pretext tasks and those of downstream tasks. This gap can result in the ''negative transfer'' problem, wherein the knowledge gained from pre-training adversely affects performance in the downstream tasks. The surge in prompt-based learning within Natural Language Processing (NLP) suggests the potential of adapting a ''pre-train, prompt'' paradigm to graphs as an alternative. However, existing graph prompting techniques are tailored to homogeneous graphs, neglecting the inherent heterogeneity of Web graphs. To bridge this gap, we propose HetGPT, a general post-training prompting framework to improve the predictive performance of pre-trained heterogeneous graph neural networks (HGNNs). The key is the design of a novel prompting function that integrates a virtual class prompt and a heterogeneous feature prompt, with the aim to reformulate downstream tasks to mirror pretext tasks. Moreover, HetGPT introduces a multi-view neighborhood aggregation mechanism, capturing the complex neighborhood structure in heterogeneous graphs. Extensive experiments on three benchmark datasets demonstrate HetGPT's capability to enhance the performance of state-of-the-art HGNNs on semi-supervised node classification.