Mitigating Overfitting in Graph Neural Networks via Feature and Hyperplane Perturbation

2022-11-28

AI Summary

Graph Neural Networks (GNNs) suffer from overfitting in the first-layer projection matrix along specific dimensions under semi-supervised learning with sparse node features, limiting their generalization on both homophilic and heterophilic graphs. To address this, we propose a joint data augmentation strategy that simultaneously perturbs initial node features and a learnable hyperplane: (i) sparse input features are augmented via random bit-flip perturbations; (ii) the hyperplane orientation is dynamically perturbed using a learnable weight matrix. This is the first work to jointly model these two complementary perturbations, departing from conventional GNN design paradigms that optimize only graph filters. Extensive experiments on multiple real-world graph benchmarks demonstrate up to a 46.5% improvement in node classification accuracy, significantly enhancing model robustness to unseen features and overall generalization capability.

Abstract

Graph neural networks (GNNs) are commonly used in semi-supervised settings. Previous research has primarily focused on finding appropriate graph filters (e.g. aggregation methods) to perform well on both homophilic and heterophilic graphs. While these methods are effective, they can still suffer from the sparsity of node features, where the initial data contain few non-zero elements. This can lead to overfitting in certain dimensions in the first projection matrix, as training samples may not cover the entire range of graph filters (hyperplanes). To address this, we propose a novel data augmentation strategy. Specifically, by flipping both the initial features and hyperplane, we create additional space for training, which leads to more precise updates of the learnable parameters and improved robustness for unseen features during inference. To the best of our knowledge, this is the first attempt to mitigate the overfitting caused by the initial features. Extensive experiments on real-world datasets show that our proposed technique increases node classification accuracy by up to 46.5% relatively.

Problem

• Graph Neural Networks
• Overfitting
• Semi-supervised Learning

Innovation

• Data Augmentation
• Graph Neural Networks
• Overfitting Prevention