Unlearnable Algorithms for In-context Learning

2024-02-01

arXiv.org

AI Summary

Efficient exact data unlearning during fine-tuning of large language models (LLMs) remains challenging due to high computational and parameter-update overhead. Method: We propose ERASE, the first context-learning–based unlearning algorithm achieving zero-parameter updates and millisecond-scale exact forgetting—scaling independently of model or dataset size. ERASE leverages prompt engineering to select few-shot examples and introduces a novel forgetting cost metric that jointly accounts for inference latency and unlearning accuracy. Contribution/Results: Theoretical analysis and empirical evaluation demonstrate that ERASE significantly reduces end-to-end forgetting cost compared to conventional fine-tuning, especially under frequent unlearning requests. This work is the first to systematically uncover the structural advantages of in-context learning for machine unlearning, establishing a new paradigm for safe and controllable LLM deployment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Machine unlearning is a desirable operation as models get increasingly deployed on data with unknown provenance. However, achieving exact unlearning -- obtaining a model that matches the model distribution when the data to be forgotten was never used -- is challenging or inefficient, often requiring significant retraining. In this paper, we focus on efficient unlearning methods for the task adaptation phase of a pretrained large language model (LLM). We observe that an LLM's ability to do in-context learning for task adaptation allows for efficient exact unlearning of task adaptation training data. We provide an algorithm for selecting few-shot training examples to prepend to the prompt given to an LLM (for task adaptation), ERASE, whose unlearning operation cost is independent of model and dataset size, meaning it scales to large models and datasets. We additionally compare our approach to fine-tuning approaches and discuss the trade-offs between the two approaches. This leads us to propose a new holistic measure of unlearning cost which accounts for varying inference costs, and conclude that in-context learning can often be more favourable than fine-tuning for deployments involving unlearning requests.

Problem

• Efficiently remove specific training data from large language models
• Achieve exact unlearning without retraining the entire model
• Reduce computational costs of unlearning while maintaining model performance

Innovation

• In-context learning replaces SGD for model adaptation
• Quantized k-means enables constant-time unlearning operations
• Method maintains performance while reducing unlearning costs