LOHA: Direct Graph Spectral Contrastive Learning Between Low-pass and High-pass Views

2025-01-06

AI Summary

Existing spectral graph neural networks for image understanding neglect semantic commonality between low-pass and high-pass filtering views, resulting in insufficient robustness of feature representations. To address this, we propose LOHA—a novel framework that pioneers the “diversity-in-harmony” principle by establishing a synergistic contrastive learning mechanism over dual spectral views (low-pass and high-pass). Specifically, LOHA designs perturbation-robust spectral trend features to align—not separate—the two views in feature space, and further refines composite representations via node-level consistency optimization. The method integrates spectral graph signal processing, self-supervised contrastive learning, and dual-band filtering modeling. Evaluated on nine heterogeneous and homogeneous graph datasets, LOHA achieves an average performance gain of 2.8% over strong baselines and surpasses fully supervised methods on multiple metrics, demonstrating superior generalization across diverse graph structures.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Spectral Graph Neural Networks effectively handle graphs with different homophily levels, with low-pass filter mining feature smoothness and high-pass filter capturing differences. When these distinct filters could naturally form two opposite views for self-supervised learning, the commonalities between the counterparts for the same node remain unexplored, leading to suboptimal performance. In this paper, a simple yet effective self-supervised contrastive framework, LOHA, is proposed to address this gap. LOHA optimally leverages low-pass and high-pass views by embracing"harmony in diversity". Rather than solely maximizing the difference between these distinct views, which may lead to feature separation, LOHA harmonizes the diversity by treating the propagation of graph signals from both views as a composite feature. Specifically, a novel high-dimensional feature named spectral signal trend is proposed to serve as the basis for the composite feature, which remains relatively unaffected by changing filters and focuses solely on original feature differences. LOHA achieves an average performance improvement of 2.8% over runner-up models on 9 real-world datasets with varying homophily levels. Notably, LOHA even surpasses fully-supervised models on several datasets, which underscores the potential of LOHA in advancing the efficacy of spectral GNNs for diverse graph structures.

Problem

• Spectral Graph Convolution
• Image Understanding
• Low-pass and High-pass Filters

Innovation

• LOHA Framework
• Spectral Signal Trend Features
• Enhanced Graph Neural Networks Performance