ScaleGNN: Towards Scalable Graph Neural Networks via Adaptive High-order Neighboring Feature Fusion

2025-04-22

AI Summary

To address the dual challenges of over-smoothing and poor scalability of Graph Neural Networks (GNNs) on large-scale graphs, this paper proposes an Adaptive High-Order Feature Fusion (AHFF) framework. Methodologically: (1) an adaptive multi-hop neighborhood fusion module dynamically aggregates information from neighbors at varying hop distances via learnable weights; (2) a redundancy masking mechanism, guided by Local Contribution Scores (LCS), suppresses inefficient high-order aggregations; and (3) a task-driven low-order enhancement strategy strengthens modeling of critical structural patterns. Evaluated on multiple large-scale real-world graph datasets, AHFF consistently outperforms state-of-the-art models—achieving an average 1.8% improvement in classification accuracy and accelerating inference by 2.3×. Notably, it is the first approach to simultaneously preserve high expressive power and enable efficient, scalable inference on massive graphs.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Graph Neural Networks (GNNs) have demonstrated strong performance across various graph-based tasks by effectively capturing relational information between nodes. These models rely on iterative message passing to propagate node features, enabling nodes to aggregate information from their neighbors. Recent research has significantly improved the message-passing mechanism, enhancing GNN scalability on large-scale graphs. However, GNNs still face two main challenges: over-smoothing, where excessive message passing results in indistinguishable node representations, especially in deep networks incorporating high-order neighbors; and scalability issues, as traditional architectures suffer from high model complexity and increased inference time due to redundant information aggregation. This paper proposes a novel framework for large-scale graphs named ScaleGNN that simultaneously addresses both challenges by adaptively fusing multi-level graph features. We first construct neighbor matrices for each order, learning their relative information through trainable weights through an adaptive high-order feature fusion module. This allows the model to selectively emphasize informative high-order neighbors while reducing unnecessary computational costs. Additionally, we introduce a High-order redundant feature masking mechanism based on a Local Contribution Score (LCS), which enables the model to retain only the most relevant neighbors at each order, preventing redundant information propagation. Furthermore, low-order enhanced feature aggregation adaptively integrates low-order and high-order features based on task relevance, ensuring effective capture of both local and global structural information without excessive complexity. Extensive experiments on real-world datasets demonstrate that our approach consistently outperforms state-of-the-art GNN models in both accuracy and computational efficiency.

Problem

• Addresses over-smoothing in deep GNNs with high-order neighbors
• Improves scalability by reducing redundant information aggregation
• Adaptively fuses multi-level graph features for efficient computation

Innovation

• Adaptive high-order feature fusion module
• High-order redundant feature masking mechanism
• Low-order enhanced feature aggregation