MAPN: Enhancing Heterogeneous Sparse Graph Representation by Mamba-based Asynchronous Aggregation

2025-02-23

AI Summary

To address three key bottlenecks in applying GNNs to large-scale heterogeneous graphs (HetGs)—over-compression, over-smoothing, and training failure on sparse graphs—this paper proposes MambaHG, the first heterogeneous graph representation learning framework built upon the state-space model (SSM) architecture Mamba. Methodologically, MambaHG introduces a meta-path-guided asynchronous multi-hop semantic aggregation mechanism: it generates node sequences via random walks, leverages Mamba to capture long-range dependencies, and incorporates hierarchical asynchronous aggregation with meta-path-aware encoding to jointly model global structure while preserving node-specific characteristics. Empirically, MambaHG achieves significant improvements in node classification and link prediction across multiple HetG benchmarks. It exhibits strong representational capacity, robustness to sparsity, and linear scalability—establishing a novel paradigm for learning on large-scale, sparse heterogeneous graphs.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Graph neural networks (GNNs) have become the state of the art for various graph-related tasks and are particularly prominent in heterogeneous graphs (HetGs). However, several issues plague this paradigm: first, the difficulty in fully utilizing long-range information, known as over-squashing; second, the tendency for excessive message-passing layers to produce indistinguishable representations, referred to as over-smoothing; and finally, the inadequacy of conventional MPNNs to train effectively on large sparse graphs. To address these challenges in deep neural networks for large-scale heterogeneous graphs, this paper introduces the Mamba-based Asynchronous Propagation Network (MAPN), which enhances the representation of heterogeneous sparse graphs. MAPN consists of two primary components: node sequence generation and semantic information aggregation. Node sequences are initially generated based on meta-paths through random walks, which serve as the foundation for a spatial state model that extracts essential information from nodes at various distances. It then asynchronously aggregates semantic information across multiple hops and layers, effectively preserving unique node characteristics and mitigating issues related to deep network degradation. Extensive experiments across diverse datasets demonstrate the effectiveness of MAPN in graph embeddings for various downstream tasks underscoring its substantial benefits for graph representation in large sparse heterogeneous graphs.

Problem

• Addressing over-squashing in GNNs
• Mitigating over-smoothing in deep networks
• Enhancing training on large sparse graphs

Innovation

• Mamba-based asynchronous propagation
• Node sequence generation
• Semantic information aggregation