CLEAR: Cluster-based Prompt Learning on Heterogeneous Graphs

2025-02-13

AI Summary

Existing heterogeneous graph prompt learning methods solely leverage node-feature prompts, neglecting the higher-order structural semantics encoded in metapaths—leading to misalignment between pretraining objectives and downstream tasks. To address this, we propose a cluster-enhanced metapath prompt learning framework: (1) for the first time, we inject learnable prompts at the metapath level, enabling cluster-aware prompting to unify pretext tasks with downstream objectives; (2) we design a metapath-enhanced GNN architecture coupled with a heterogeneous graph reconstruction-based pretraining paradigm, explicitly modeling higher-order semantic associations. Evaluated on multiple heterogeneous graph node classification benchmarks, our method consistently outperforms state-of-the-art approaches, achieving up to a 5.0% improvement in F1 score. This demonstrates that structural-aware prompting significantly enhances task alignment and transferability in heterogeneous graph representation learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Search and Optimization: Metareasoning and Metaheuristics
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Prompt learning has attracted increasing attention in the graph domain as a means to bridge the gap between pretext and downstream tasks. Existing studies on heterogeneous graph prompting typically use feature prompts to modify node features for specific downstream tasks, which do not concern the structure of heterogeneous graphs. Such a design also overlooks information from the meta-paths, which are core to learning the high-order semantics of the heterogeneous graphs. To address these issues, we propose CLEAR, a Cluster-based prompt LEARNING model on heterogeneous graphs. We present cluster prompts that reformulate downstream tasks as heterogeneous graph reconstruction. In this way, we align the pretext and downstream tasks to share the same training objective. Additionally, our cluster prompts are also injected into the meta-paths such that the prompt learning process incorporates high-order semantic information entailed by the meta-paths. Extensive experiments on downstream tasks confirm the superiority of CLEAR. It consistently outperforms state-of-the-art models, achieving up to 5% improvement on the F1 metric for node classification.

Problem

• Enhance heterogeneous graph learning
• Integrate meta-path semantic information
• Improve downstream task performance

Innovation

• Cluster-based prompt learning
• Heterogeneous graph reconstruction
• Meta-path semantic incorporation