Exact Certification of (Graph) Neural Networks Against Label Poisoning

2024-11-30

arXiv.org

AI Summary

This work addresses the challenge of certifying robustness of Graph Neural Networks (GNNs) against label poisoning attacks—where training labels are maliciously flipped. We propose the **first exact, computationally tractable sample-wise and population-wise robustness certificates** for this setting. Methodologically, we model wide-GNN training dynamics via the Neural Tangent Kernel (NTK) and reformulate the bilevel robustness verification problem as a Mixed-Integer Linear Program (MILP), supporting diverse GNN architectures. Theoretically and empirically, we identify a “robustness plateau” under moderate perturbation budgets and quantify how architectural choices—including activation functions, depth, and skip connections—exert directional effects on certified robustness. Extensive evaluation across standard graph benchmarks demonstrates both the effectiveness and tightness of our certificates. To our knowledge, this establishes the first exact certification framework for label poisoning in general neural networks.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Machine learning models are highly vulnerable to label flipping, i.e., the adversarial modification (poisoning) of training labels to compromise performance. Thus, deriving robustness certificates is important to guarantee that test predictions remain unaffected and to understand worst-case robustness behavior. However, for Graph Neural Networks (GNNs), the problem of certifying label flipping has so far been unsolved. We change this by introducing an exact certification method, deriving both sample-wise and collective certificates. Our method leverages the Neural Tangent Kernel (NTK) to capture the training dynamics of wide networks enabling us to reformulate the bilevel optimization problem representing label flipping into a Mixed-Integer Linear Program (MILP). We apply our method to certify a broad range of GNN architectures in node classification tasks. Thereby, concerning the worst-case robustness to label flipping: $(i)$ we establish hierarchies of GNNs on different benchmark graphs; $(ii)$ quantify the effect of architectural choices such as activations, depth and skip-connections; and surprisingly, $(iii)$ uncover a novel phenomenon of the robustness plateauing for intermediate perturbation budgets across all investigated datasets and architectures. While we focus on GNNs, our certificates are applicable to sufficiently wide NNs in general through their NTK. Thus, our work presents the first exact certificate to a poisoning attack ever derived for neural networks, which could be of independent interest.

Problem

• Certify robustness of Graph Neural Networks against label flipping attacks.
• Develop exact certification method using Neural Tangent Kernel and MILP.
• Analyze worst-case robustness and architectural impacts on GNN performance.

Innovation

• Exact certification method for GNNs against label poisoning
• Uses Neural Tangent Kernel for training dynamics
• Reformulates label flipping into Mixed-Integer Linear Program