On the Recoverability of Private Information Unlearning in Large Language Models

2026-08-30

AI Summary

本文针对大型语言模型记忆敏感信息的问题，通过构建合成数据集和白盒审计框架评估五种现有遗忘方法的有效性，揭示当前方法常无法彻底移除敏感信息。

Abstract

Large language models (LLMs) can memorize sensitive information, raising serious privacy concerns. Machine unlearning offers a potential solution to remove such information, but it remains unclear whether existing methods truly erase it or merely hide it within the model. A key challenge is quantifying the persistence of sensitive data under a unified evaluation framework. To address this, we construct a synthetic dataset containing fake private information and propose a white-box auditing framework to systematically assess whether claimed-forgotten information is genuinely removed. Using this framework, we evaluate five existing unlearning methods and find that a simple "inverse greedy" decoding -- selecting the least likely token at each step -- can recover supposedly forgotten private information. Our results reveal that current unlearning approaches often fail to fully eliminate sensitive information, highlighting the need for more reliable methods to ensure privacy in deployed LLMs.

Problem

• Large Language Models
• Machine Unlearning
• Sensitive Information
• Privacy Concerns
• Evaluation Framework

Innovation

• Machine Unlearning
• White-box Auditing Framework
• Inverse Greedy Decoding