Pruning as a Defense: Reducing Memorization in Large Language Models
Large language models (LLMs) inherently memorize training data, posing significant privacy risks—including data leakage and membership inference attacks. This work systematically demonstrates, for the first time, that structured pruning and sparsification—guided by weight importance estimation—effectively suppress LLMs’ memorization behavior, establishing “pruning-as-defense” as a novel privacy-preserving paradigm. Evaluated across multiple mainstream LLMs, our approach reduces memory leakage rates by 40–65% while retaining over 90% of original task performance. Crucially, experiments conducted under standardized membership inference attack benchmarks confirm that pruning is not merely a model compression technique but serves as a lightweight, general-purpose, and retraining-free foundational defense mechanism for privacy enhancement.