Norm Augmented Graph AutoEncoders for Link Prediction

2025-02-09

AI Summary

Graph Autoencoders (GAEs) exhibit significant degree bias in link prediction: performance on low-degree nodes is substantially worse than on high-degree nodes. This work identifies, for the first time, that embedding norm strongly correlates with node degree—and that this correlation is a key mechanism underlying the bias. To address it, we propose a lightweight, architecture-agnostic “self-loop augmentation” strategy: explicitly adding self-loops to low-degree nodes in the training objective to increase their embedding norms. Our method requires no modifications to the model architecture or decoder design, and is fully compatible with mainstream GNN backbones and training paradigms. On multiple benchmark datasets, it improves link prediction AUC for low-degree nodes by 2.1–4.7 percentage points, yielding substantial overall performance gains with negligible computational overhead.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Link Prediction (LP) is a crucial problem in graph-structured data. Graph Neural Networks (GNNs) have gained prominence in LP, with Graph AutoEncoders (GAEs) being a notable representation. However, our empirical findings reveal that GAEs' LP performance suffers heavily from the long-tailed node degree distribution, i.e., low-degree nodes tend to exhibit inferior LP performance compared to high-degree nodes. emph{What causes this degree-related bias, and how can it be mitigated?} In this study, we demonstrate that the norm of node embeddings learned by GAEs exhibits variation among nodes with different degrees, underscoring its central significance in influencing the final performance of LP. Specifically, embeddings with larger norms tend to guide the decoder towards predicting higher scores for positive links and lower scores for negative links, thereby contributing to superior performance. This observation motivates us to improve GAEs' LP performance on low-degree nodes by increasing their embedding norms, which can be implemented simply yet effectively by introducing additional self-loops into the training objective for low-degree nodes. This norm augmentation strategy can be seamlessly integrated into existing GAE methods with light computational cost. Extensive experiments on various datasets and GAE methods show the superior performance of norm-augmented GAEs.

Problem

• Addressing degree-related bias in link prediction
• Improving performance for low-degree nodes in graphs
• Enhancing Graph AutoEncoders with norm augmentation

Innovation

• Augments node embedding norms
• Introduces self-loops strategically
• Improves low-degree node performance