ESC: Erasing Space Concept for Knowledge Deletion

2025-04-03

AI Summary

Deep learning models often retain user-specific private knowledge in their embeddings, making complete and verifiable erasure difficult and posing privacy leakage risks. Method: This paper formally defines the “Knowledge Deletion (KD)” task and introduces the Knowledge Retention score (KR) as a quantitative evaluation metric. We propose ESC, a retraining-free subspace erasure mechanism, and extend it to ESC-T—a learnable binary mask that jointly enforces feature activation suppression and subspace importance constraints to achieve precise, functionality-preserving forgetting. Contribution/Results: Extensive experiments across multiple datasets, architectures (e.g., ResNet, ViT), and sensitive domains—including face recognition—demonstrate state-of-the-art forgetting performance. ESC-T exhibits strong generalization across model architectures and downstream tasks while maintaining predictive accuracy on retained knowledge, offering a practical, scalable solution for privacy-compliant model updates.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

As concerns regarding privacy in deep learning continue to grow, individuals are increasingly apprehensive about the potential exploitation of their personal knowledge in trained models. Despite several research efforts to address this, they often fail to consider the real-world demand from users for complete knowledge erasure. Furthermore, our investigation reveals that existing methods have a risk of leaking personal knowledge through embedding features. To address these issues, we introduce a novel concept of Knowledge Deletion (KD), an advanced task that considers both concerns, and provides an appropriate metric, named Knowledge Retention score (KR), for assessing knowledge retention in feature space. To achieve this, we propose a novel training-free erasing approach named Erasing Space Concept (ESC), which restricts the important subspace for the forgetting knowledge by eliminating the relevant activations in the feature. In addition, we suggest ESC with Training (ESC-T), which uses a learnable mask to better balance the trade-off between forgetting and preserving knowledge in KD. Our extensive experiments on various datasets and models demonstrate that our proposed methods achieve the fastest and state-of-the-art performance. Notably, our methods are applicable to diverse forgetting scenarios, such as facial domain setting, demonstrating the generalizability of our methods. The code is available at http://github.com/KU-VGI/ESC .

Problem

• Addressing privacy concerns in deep learning by complete knowledge erasure
• Preventing personal knowledge leakage through embedding features
• Balancing knowledge forgetting and preservation in diverse scenarios

Innovation

• Introduces Knowledge Deletion (KD) concept
• Proposes training-free Erasing Space Concept (ESC)
• Suggests ESC with Training (ESC-T) for balance