Adversarial Training for Graph Neural Networks via Graph Subspace Energy Optimization

2024-12-25

AI Summary

Graph neural networks (GNNs) exhibit insufficient robustness against topological perturbations in inductive learning. To address this, we propose Graph Subspace Energy (GSE), a novel robustness metric, and design AT-GSE—an adversarial training framework grounded in GSE. AT-GSE is the first method to leverage GSE for guiding adversarial perturbation generation, enabling joint modeling of local and global topological perturbations. It incorporates GSE-based regularization and an efficient perturbation generation strategy utilizing LRBCD/PRBCD optimization, accelerated via randomized SVD and Nyström low-rank approximations. Extensive experiments demonstrate that AT-GSE significantly improves adversarial accuracy on both homophilic and heterophilic datasets, while simultaneously surpassing baseline models in clean accuracy—thereby enhancing both robustness and generalization.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Despite impressive capability in learning over graph-structured data, graph neural networks (GNN) suffer from adversarial topology perturbation in both training and inference phases. While adversarial training has demonstrated remarkable effectiveness in image classification tasks, its suitability for GNN models has been doubted until a recent advance that shifts the focus from transductive to inductive learning. Still, GNN robustness in the inductive setting is under-explored, and it calls for deeper understanding of GNN adversarial training. To this end, we propose a new concept of graph subspace energy (GSE) -- a generalization of graph energy that measures graph stability -- of the adjacency matrix, as an indicator of GNN robustness against topology perturbations. To further demonstrate the effectiveness of such concept, we propose an adversarial training method with the perturbed graphs generated by maximizing the GSE regularization term, referred to as AT-GSE. To deal with the local and global topology perturbations raised respectively by LRBCD and PRBCD, we employ randomized SVD (RndSVD) and Nystrom low-rank approximation to favor the different aspects of the GSE terms. An extensive set of experiments shows that AT-GSE outperforms consistently the state-of-the-art GNN adversarial training methods over different homophily and heterophily datasets in terms of adversarial accuracy, whilst more surprisingly achieving a superior clean accuracy on non-perturbed graphs.

Problem

• Graph Neural Networks
• Stability
• Adversarial Environment

Innovation

• Graph Subspace Energy
• Adversarial Training
• Robust GNN