Learning Personalized Scoping for Graph Neural Networks under Heterophily

2024-09-11

arXiv.org

AI Summary

Deep graph neural networks (GNNs) suffer performance degradation on heterogeneous graphs due to the failure of the homophily assumption, leading to suboptimal neighborhood aggregation. Method: This paper proposes Adaptive Scope (AS), the first framework that models personalized receptive fields as a lightweight depth classification task—decoupling scope prediction from feature learning and eliminating node-wise learnable weights to mitigate overfitting. AS designs a depth classifier grounded in structural pattern encoding and multi-hop neighborhood statistics, enabling dynamic routing to the optimal aggregation depth during GNN inference. Contribution/Results: Extensive experiments across十余 heterogeneous graph benchmarks and diverse GNN backbones demonstrate that AS consistently improves node classification accuracy, exhibits strong generalization, and is architecture-agnostic—effectively alleviating the performance drop typically induced by increasing GNN depth.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Deep Neural Architectures and Foundation Models
• Multiagent Systems: Adversarial Agents

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Heterophilous graphs, where dissimilar nodes tend to connect, pose a challenge for graph neural networks (GNNs) as their superior performance typically comes from aggregating homophilous information. Increasing the GNN depth can expand the scope (i.e., receptive field), potentially finding homophily from the higher-order neighborhoods. However, uniformly expanding the scope results in subpar performance since real-world web graphs often exhibit homophily disparity between nodes. An ideal way is personalized scopes, allowing nodes to have varying scope sizes. Existing methods typically add node-adaptive weights for each hop. Although expressive, they inevitably suffer from severe overfitting. To address this issue, we formalize personalized scoping as a separate scope classification problem that overcomes GNN overfitting in node classification. Specifically, we predict the optimal GNN depth for each node. Our theoretical and empirical analysis suggests that accurately predicting the depth can significantly enhance generalization. We further propose Adaptive Scope (AS), a lightweight approach that only participates in GNN inference. AS encodes structural patterns and predicts the depth to select the best model for each node's prediction. Experimental results show that AS is highly flexible with various GNN architectures across a wide range of datasets while significantly improving accuracy.

Problem

• Heterophilous graphs challenge GNNs due to dissimilar node connections
• Deeper GNNs face performance degradation despite expanded receptive fields
• Generalization disparity arises between shallow and deep GNN variants

Innovation

• Mixture of scope experts enhances GNN generalization
• Combines shallow and deep GNNs for heterophilous graphs
• Improves accuracy across diverse datasets flexibly