Answer When Needed, Forget When Not: Language Models Pretend to Forget via In-Context Knowledge Unlearning

2024-10-01

arXiv.org

AI Summary

To address the need for dynamic, context-sensitive isolation of sensitive knowledge in multi-stakeholder scenarios involving large language models (LLMs), this paper introduces “contextual knowledge forgetting”—a novel paradigm wherein the model autonomously triggers selective forgetting during inference based on query context (e.g., user permission tokens), enabling authorized users to access sensitive knowledge while rendering it inaccessible to unauthorized ones. Methodologically, we propose a context-aware instruction-tuning strategy and conduct inter-layer behavioral analysis, uncovering an intrinsic mechanism whereby the model actively “simulates forgetting” at its final hidden layer. Evaluated on TOFU and AGE benchmarks using Llama2-7B/13B and Mistral-7B, our approach achieves 95% forgetting accuracy and 80% retention of unrelated knowledge—substantially outperforming existing baselines. This work constitutes the first systematic formalization and realization of fine-grained, test-time, context-driven knowledge forgetting.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

As large language models (LLMs) are applied across diverse domains, the ability to selectively unlearn specific information has become increasingly essential. For instance, LLMs are expected to provide confidential information to authorized internal users, such as employees or trusted partners, while withholding it from external users, including the general public and unauthorized entities. In response to this challenge, we propose a novel method termed ``in-context knowledge unlearning'', which enables the model to selectively forget information in test-time based on the context of the query. Our method fine-tunes pre-trained LLMs to enable prompt unlearning of target knowledge within the context, while preserving other knowledge. Experiments on the TOFU and AGE datasets using Llama2-7B/13B and Mistral-7B models show our method achieves up to 95% forgetting accuracy while retaining 80% of unrelated knowledge, significantly outperforming baselines in both in-domain and out-of-domain scenarios. Further investigation into the model's internal behavior revealed that while fine-tuned LLMs generate correct predictions in the middle layers and maintain them up to the final layer, they make the decision to forget at the last layer, i.e., ``LLMs pretend to forget''. Our findings offer valuable insights into enhancing the robustness of unlearning mechanisms in LLMs, setting a foundation for future research in the field.

Problem

• Enables selective forgetting of specific information in LLMs
• Fine-tunes LLMs to forget target knowledge contextually
• Improves robustness of unlearning mechanisms in language models

Innovation

• In-context knowledge unlearning for selective forgetting
• Fine-tunes LLMs to forget target knowledge contextually
• Achieves high forget accuracy while preserving unrelated knowledge