Mitigating Degree Bias in Graph Representation Learning with Learnable Structural Augmentation and Structural Self-Attention

2025-04-21

AI Summary

Graph neural networks suffer from degree bias due to long-tailed degree distributions—high-degree nodes dominate message passing, while low-degree nodes suffer from insufficient information propagation and poor representation learning. To address this, we propose Degree-Fair Graph Transformer (DegFairGT), the first model integrating *intra-community role-aware learnable structural enhancement* with *structural self-attention*. It selectively incorporates role-similar non-neighboring nodes to enrich low-degree node representations while suppressing overloaded propagation from high-degree nodes. Additionally, we introduce a *p-step transition probability-based self-supervised regularization*, jointly optimizing structural fidelity and degree fairness. Extensive experiments across six benchmark datasets demonstrate that DegFairGT significantly improves degree fairness and consistently outperforms state-of-the-art methods on both node classification and clustering tasks.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Graph Neural Networks (GNNs) update node representations through message passing, which is primarily based on the homophily principle, assuming that adjacent nodes share similar features. However, in real-world graphs with long-tailed degree distributions, high-degree nodes dominate message passing, causing a degree bias where low-degree nodes remain under-represented due to inadequate messages. The main challenge in addressing degree bias is how to discover non-adjacent nodes to provide additional messages to low-degree nodes while reducing excessive messages for high-degree nodes. Nevertheless, exploiting non-adjacent nodes to provide valuable messages is challenging, as it could generate noisy information and disrupt the original graph structures. To solve it, we propose a novel Degree Fairness Graph Transformer, named DegFairGT, to mitigate degree bias by discovering structural similarities between non-adjacent nodes through learnable structural augmentation and structural self-attention. Our key idea is to exploit non-adjacent nodes with similar roles in the same community to generate informative edges under our augmentation, which could provide informative messages between nodes with similar roles while ensuring that the homophily principle is maintained within the community. To enable DegFairGT to learn such structural similarities, we then propose a structural self-attention to capture the similarities between node pairs. To preserve global graph structures and prevent graph augmentation from hindering graph structure, we propose a Self-Supervised Learning task to preserve p-step transition probability and regularize graph augmentation. Extensive experiments on six datasets showed that DegFairGT outperformed state-of-the-art baselines in degree fairness analysis, node classification, and node clustering tasks.

Problem

• Addressing degree bias in graph neural networks
• Enhancing representation of low-degree nodes
• Balancing message passing for diverse node degrees

Innovation

• Learnable structural augmentation for node similarity
• Structural self-attention capturing node pair similarities
• Self-supervised learning preserving graph transition probability