Mitigating Memorization In Language Models

2024-10-03

arXiv.org

AI Summary

To mitigate the risk of sensitive information leakage due to excessive memorization of training data in large language models (LLMs), this paper proposes a systematic machine unlearning framework. Methodologically, it introduces five novel unlearning algorithms, including BalancedSubnet—the first gradient-editing-based approach incorporating subnet rebalancing—to enable precise identification and selective removal of memorized samples; it further develops the lightweight TinyMem model family to facilitate efficient algorithm development and evaluation. The key contributions are: (1) BalancedSubnet achieves state-of-the-art performance across multiple benchmarks, completely erasing target memories while incurring an average task accuracy drop of less than 1.2%; and (2) the proposed techniques generalize seamlessly to production-grade models—including LLaMA-2 and Qwen—demonstrating strong practicality and scalability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Language models (LMs) can"memorize"information, i.e., encode training data in their weights in such a way that inference-time queries can lead to verbatim regurgitation of that data. This ability to extract training data can be problematic, for example, when data are private or sensitive. In this work, we investigate methods to mitigate memorization: three regularizer-based, three finetuning-based, and eleven machine unlearning-based methods, with five of the latter being new methods that we introduce. We also introduce TinyMem, a suite of small, computationally-efficient LMs for the rapid development and evaluation of memorization-mitigation methods. We demonstrate that the mitigation methods that we develop using TinyMem can successfully be applied to production-grade LMs, and we determine via experiment that: regularizer-based mitigation methods are slow and ineffective at curbing memorization; fine-tuning-based methods are effective at curbing memorization, but overly expensive, especially for retaining higher accuracies; and unlearning-based methods are faster and more effective, allowing for the precise localization and removal of memorized information from LM weights prior to inference. We show, in particular, that our proposed unlearning method BalancedSubnet outperforms other mitigation methods at removing memorized information while preserving performance on target tasks.

Problem

• Language Model
• Privacy Preservation
• Information Forgetting

Innovation

• BalancedSubnet
• InformationForgetting
• TinyMemModel