Echoless Label-Based Pre-computation for Memory-Efficient Heterogeneous Graph Learning

2025-11-14

AI Summary

Existing label-precomputed heterogeneous graph neural networks (HGNNs) suffer from training label leakage—termed the “echo effect”—during multi-hop message passing, which severely impairs model generalization. Current mitigation strategies either incur prohibitive memory overhead or lack compatibility with advanced message-passing paradigms. To address this, we propose **Echo-Free Propagation (EFP)**, the first propagation mechanism featuring a partition-focused strategy: asymmetric node partitioning enforces inter-partition message isolation, while label pre-extraction and distribution calibration jointly eliminate echo propagation without compromising information integrity. EFP is memory-efficient, broadly applicable, and seamlessly integrates with diverse message-passing architectures. Extensive experiments on multiple public benchmarks demonstrate that EFP reduces memory consumption by an average of 37% while outperforming state-of-the-art baselines in accuracy—achieving up to a 2.1% absolute gain.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Heterogeneous Graph Neural Networks (HGNNs) are widely used for deep learning on heterogeneous graphs. Typical end-to-end HGNNs require repetitive message passing during training, limiting efficiency for large-scale real-world graphs. Pre-computation-based HGNNs address this by performing message passing only once during preprocessing, collecting neighbor information into regular-shaped tensors, which enables efficient mini-batch training. Label-based pre-computation methods collect neighbors'label information but suffer from training label leakage, where a node's own label information propagates back to itself during multi-hop message passing - the echo effect. Existing mitigation strategies are memory-inefficient on large graphs or suffer from compatibility issues with advanced message passing methods. We propose Echoless Label-based Pre-computation (Echoless-LP), which eliminates training label leakage with Partition-Focused Echoless Propagation (PFEP). PFEP partitions target nodes and performs echoless propagation, where nodes in each partition collect label information only from neighbors in other partitions, avoiding echo while remaining memory-efficient and compatible with any message passing method. We also introduce an Asymmetric Partitioning Scheme (APS) and a PostAdjust mechanism to address information loss from partitioning and distributional shifts across partitions. Experiments on public datasets demonstrate that Echoless-LP achieves superior performance and maintains memory efficiency compared to baselines.

Problem

• HGNNs suffer from inefficient repetitive message passing during training
• Label-based pre-computation causes training label leakage through echo effects
• Existing solutions have memory inefficiency and compatibility limitations

Innovation

• Echoless propagation prevents training label leakage
• Partition-focused method maintains memory efficiency
• Asymmetric partitioning addresses information loss issues