Beyond Node Attention: Multi-Scale Harmonic Encoding for Feature-Wise Graph Message Passing

2025-05-21

AI Summary

Traditional GNNs aggregate neighbor embeddings as holistic vectors, limiting their capacity to model feature-level and direction-sensitive message passing. To address this, we propose MSH-GNN, a novel architecture that enables frequency-domain adaptive neighborhood feature fusion via node-specific harmonic projections. It further incorporates multi-scale sinusoidal encoding modulation and spectrum-aware attention pooling. Theoretically, MSH-GNN achieves expressive power equivalent to the 1-WL test and approximates translation-invariant kernels. Crucially, it is the first framework to deeply integrate harmonic graph signal processing with feature-level message passing. Empirically, MSH-GNN significantly outperforms state-of-the-art methods on graph and node classification benchmarks. Notably, it exhibits superior discriminative capability under joint topological–spectral variations and structural asymmetries—challenges where conventional GNNs often falter.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Social mining and social search on the Web

Abstract

Conventional Graph Neural Networks (GNNs) aggregate neighbor embeddings as holistic vectors, lacking the ability to identify fine-grained, direction-specific feature relevance. We propose MSH-GNN (Multi-Scale Harmonic Graph Neural Network), a novel architecture that performs feature-wise adaptive message passing through node-specific harmonic projections. For each node, MSH-GNN dynamically projects neighbor features onto frequency-sensitive directions determined by the target node's own representation. These projections are further modulated using learnable sinusoidal encodings at multiple frequencies, enabling the model to capture both smooth and oscillatory structural patterns across scales. A frequency-aware attention pooling mechanism is introduced to emphasize spectrally and structurally salient nodes during readout. Theoretically, we prove that MSH-GNN approximates shift-invariant kernels and matches the expressive power of the 1-Weisfeiler-Lehman (1-WL) test. Empirically, MSH-GNN consistently outperforms state-of-the-art models on a wide range of graph and node classification tasks. Furthermore, in challenging classification settings involving joint variations in graph topology and spectral frequency, MSH-GNN excels at capturing structural asymmetries and high-frequency modulations, enabling more accurate graph discrimination.

Problem

• Enables feature-wise adaptive message passing in GNNs
• Captures multi-scale structural patterns via harmonic projections
• Improves graph discrimination with frequency-aware attention pooling

Innovation

• Feature-wise adaptive message passing via harmonic projections
• Multi-scale learnable sinusoidal encodings for structural patterns
• Frequency-aware attention pooling for salient node readout