Generative-Contrastive Heterogeneous Graph Neural Network

2024-04-03

arXiv.org

AI Summary

To address the limitations of data augmentation scarcity, severe bias in negative sampling, and insufficient modeling of local heterogeneous structural information in heterogeneous graph contrastive learning, this paper proposes a generative-contrastive joint learning framework. We introduce the first generative contrastive learning paradigm for heterogeneous graphs, designing masked autoencoder-based view augmentation to enhance data diversity. To mitigate discriminator bias, we propose a position- and semantics-aware hard negative sampling strategy. Furthermore, we formulate a hierarchical local-global contrastive loss that explicitly captures heterogeneous structures at three granularities: node-level, meta-path-level, and subgraph-level. Extensive experiments on eight real-world heterogeneous graph datasets demonstrate that our method consistently outperforms 17 state-of-the-art baselines across node classification and link prediction tasks, achieving new state-of-the-art performance.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Heterogeneous Graphs (HGs) effectively model complex relationships in the real world through multi-type nodes and edges. In recent years, inspired by self-supervised learning (SSL), contrastive learning (CL)-based Heterogeneous Graphs Neural Networks (HGNNs) have shown great potential in utilizing data augmentation and contrastive discriminators for downstream tasks. However, data augmentation remains limited due to the graph data's integrity. Furthermore, the contrastive discriminators suffer from sampling bias and lack local heterogeneous information. To tackle the above limitations, we propose a novel Generative-Contrastive Heterogeneous Graph Neural Network (GC-HGNN). Specifically, we propose a heterogeneous graph generative learning method that enhances CL-based paradigm. This paradigm includes: 1) A contrastive view augmentation strategy using a masked autoencoder. 2) Position-aware and semantics-aware positive sample sampling strategy for generating hard negative samples. 3) A hierarchical contrastive learning strategy aimed at capturing local and global information. Furthermore, the hierarchical contrastive learning and sampling strategies aim to constitute an enhanced contrastive discriminator under the generative-contrastive perspective. Finally, we compare our model with seventeen baselines on eight real-world datasets. Our model outperforms the latest baselines on node classification and link prediction tasks.

Problem

• Limited data augmentation in heterogeneous graph neural networks
• Sampling bias in contrastive discriminators for heterogeneous graphs
• Lack of local heterogeneous information in contrastive learning

Innovation

• Masked autoencoder for contrastive view augmentation
• Position-aware and semantics-aware positive sampling
• Hierarchical contrastive learning for local-global information