Streamlined Federated Unlearning: Unite as One to Be Highly Efficient

2024-11-28

arXiv.org

AI Summary

To address the “right to be forgotten” compliance requirement in federated learning, existing federated unlearning (FU) methods struggle to simultaneously ensure thorough forgetting, model performance preservation, and resource efficiency. This paper proposes SFU, a lightweight federated unlearning framework: it introduces the first multi-teacher collaborative knowledge distillation mechanism, integrated with gradient correction and a lightweight synchronization protocol, enabling precise removal of target data influence without full retraining. SFU significantly improves computational, communication, and storage efficiency, and supports cross-modal tasks (e.g., image and text). Experiments on mainstream benchmarks demonstrate that SFU achieves lossless unlearning accuracy, reduces time and communication overhead by over 60% compared to complete retraining, and exhibits robustness against backdoor attacks. Thus, SFU establishes a new paradigm for efficient, secure, and regulation-compliant federated unlearning.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Recently, the enactment of ``right to be forgotten"laws and regulations has imposed new privacy requirements on federated learning (FL). Researchers aim to remove the influence of certain data from the trained model without training from scratch through federated unlearning (FU). While current FU research has shown progress in enhancing unlearning efficiency, it often results in degraded model performance upon achieving the goal of data unlearning, necessitating additional steps to recover the performance of the unlearned model. Moreover, these approaches also suffer from many shortcomings such as high consumption of computational and storage resources. To this end, we propose a streamlined federated unlearning approach (SFU) aimed at effectively removing the influence of the target data while preserving the model performance on the retained data without degradation. We design a practical multi-teacher system that achieves both target data influence removal and model performance preservation by guiding the unlearned model through several distinct teacher models. SFU is both computationally and storage-efficient, highly flexible, and generalizable. We conduct extensive experiments on both image and text benchmark datasets. The results demonstrate that SFU significantly improves time and communication efficiency compared to the benchmark retraining method and significantly outperforms existing SOTA methods. Additionally, we verify the effectiveness of SFU using the backdoor attack.

Problem

• Efficiently remove target data influence in federated learning
• Preserve model performance without additional recovery steps
• Reduce computational and storage resource consumption

Innovation

• Streamlined federated unlearning for efficiency
• Multi-teacher system preserves model performance
• Computationally and storage-efficient unlearning approach