Towards Source-Free Machine Unlearning

2025-08-20

AI Summary

This paper addresses machine unlearning under the source-free setting—safely removing specific private or copyright-protected data samples from a pre-trained model without access to the original training data. We propose the first zero-shot machine unlearning paradigm: leveraging only black-box model queries to estimate the Hessian matrix of the remaining data, then applying optimization-theoretic techniques to perform efficient and verifiable unlearning updates. Our method provides rigorous theoretical guarantees, including gradient consistency and bounded distributional shift after unlearning. Extensive experiments on multiple benchmark datasets demonstrate high unlearning accuracy and robust preservation of model utility. The core contribution is breaking the long-standing dependency of unlearning methods on original training data, achieving—for the first time—source-free, zero-sample, and strongly theoretically grounded machine unlearning.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

As machine learning becomes more pervasive and data privacy regulations evolve, the ability to remove private or copyrighted information from trained models is becoming an increasingly critical requirement. Existing unlearning methods often rely on the assumption of having access to the entire training dataset during the forgetting process. However, this assumption may not hold true in practical scenarios where the original training data may not be accessible, i.e., the source-free setting. To address this challenge, we focus on the source-free unlearning scenario, where an unlearning algorithm must be capable of removing specific data from a trained model without requiring access to the original training dataset. Building on recent work, we present a method that can estimate the Hessian of the unknown remaining training data, a crucial component required for efficient unlearning. Leveraging this estimation technique, our method enables efficient zero-shot unlearning while providing robust theoretical guarantees on the unlearning performance, while maintaining performance on the remaining data. Extensive experiments over a wide range of datasets verify the efficacy of our method.

Problem

• Addressing source-free machine unlearning without original data access
• Removing specific private or copyrighted information from trained models
• Estimating Hessian of unknown remaining data for efficient unlearning

Innovation

• Source-free unlearning without original data
• Estimating Hessian for remaining training data
• Zero-shot unlearning with theoretical guarantees