PrivUn: Unveiling Latent Ripple Effects and Shallow Forgetting in Privacy Unlearning

2026-04-23

AI Summary

Large language models are prone to memorizing private data, yet existing unlearning methods lack robustness under realistic attacks and often suffer from superficial forgetting and gradient-driven implicit ripple effects. This work proposes a three-tier privacy attack evaluation framework—encompassing direct retrieval, context recovery, and fine-tuning restoration—combined with metrics for unlearning score, association strength, and forgetting depth to systematically assess unlearning robustness. To address these limitations, the study introduces a novel approach that leverages gradient similarity for core-set selection and enforces multi-layer representation constraints, thereby shifting unlearning from surface-level to deep semantic removal. Experimental results demonstrate that the proposed method significantly enhances the thorough erasure of private information in deeper network layers and substantially improves the model’s resilience against sophisticated privacy attacks.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Large language models (LLMs) often memorize private information during training, raising serious privacy concerns. While machine unlearning has emerged as a promising solution, its true effectiveness against privacy attacks remains unclear. To address this, we propose PrivUn, a new evaluation framework that systematically assesses unlearning robustness through three-tier attack scenarios: direct retrieval, in-context learning recovery, and fine-tuning restoration; combined with quantitative analysis using forgetting scores, association metrics, and forgetting depth assessment. Our study exposes significant weaknesses in current unlearning methods, revealing two key findings: 1) unlearning exhibits gradient-driven ripple effects: unlike traditional forgetting which follows semantic relations (e.g., knowledge graphs), privacy unlearning propagates across latent gradient-based associations; and 2) most methods suffer from shallow forgetting, failing to remove private information distributed across multiple deep model layers. To validate these insights, we explore two strategies: association-aware core-set selection that leverages gradient similarity, and multi-layer deep intervention through representational constraints. These strategies represent a paradigm shift from shallow forgetting to deep forgetting.

Problem

• privacy unlearning
• shallow forgetting
• ripple effects
• large language models
• private information

Innovation

• privacy unlearning
• gradient-driven ripple effects
• shallow forgetting
• deep forgetting
• association-aware core-set selection