Scalable and Certifiable Graph Unlearning: Overcoming the Approximation Error Barrier

2024-08-17

AI Summary

Existing certified graph unlearning methods for large-scale Graph Neural Networks (GNNs) suffer from poor scalability—requiring full-graph propagation for each unlearning request—rendering them infeasible for billion-edge graphs. Method: This work introduces the first certified unlearning framework incorporating *error-bounded approximate graph propagation*, integrating randomized walk truncation, localized neighborhood reweighting, and error propagation modeling. It further combines differential privacy–driven certified bound analysis with an incremental embedding update mechanism. Contribution/Results: The approach guarantees bounded node embedding error under strict (ε,δ)-certified unlearning. On ogbn-papers100M (≈1B edges), unlearning 5,000 edges takes only 20 seconds (5 seconds for embedding updates), achieving a 340× speedup over full retraining—overcoming the longstanding scalability bottleneck in certified graph unlearning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Graph unlearning has emerged as a pivotal research area for ensuring privacy protection, given the widespread adoption of Graph Neural Networks (GNNs) in applications involving sensitive user data. Among existing studies, certified graph unlearning is distinguished by providing robust privacy guarantees. However, current certified graph unlearning methods are impractical for large-scale graphs because they necessitate the costly re-computation of graph propagation for each unlearning request. Although numerous scalable techniques have been developed to accelerate graph propagation for GNNs, their integration into certified graph unlearning remains uncertain as these scalable approaches introduce approximation errors into node embeddings. In contrast, certified graph unlearning demands bounded model error on exact node embeddings to maintain its certified guarantee. To address this challenge, we present ScaleGUN, the first approach to scale certified graph unlearning to billion-edge graphs. ScaleGUN integrates the approximate graph propagation technique into certified graph unlearning, offering certified guarantees for three unlearning scenarios: node feature, edge, and node unlearning. Extensive experiments on real-world datasets demonstrate the efficiency and unlearning efficacy of ScaleGUN. Remarkably, ScaleGUN accomplishes $(epsilon,delta)=(1,10^{-4})$ certified unlearning on the billion-edge graph ogbn-papers100M in 20 seconds for a 5,000 random edge removal request -- of which only 5 seconds are required for updating the node embeddings -- compared to 1.91 hours for retraining and 1.89 hours for re-propagation. Our code is available at https://github.com/luyi256/ScaleGUN.

Problem

• Addresses scalability in certified graph unlearning
• Overcomes approximation error in scalable techniques
• Ensures privacy in large-scale graph applications

Innovation

• Scalable certified graph unlearning
• Approximate graph propagation technique
• Efficient billion-edge graph processing