CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge

2026-04-16

AI Summary

Pretraining data for large language models inevitably contain sensitive content that is difficult to fully filter, necessitating efficient and continual unlearning mechanisms that preserve model utility. This work proposes a real-time, continual unlearning approach that operates without modifying model parameters: it employs a dedicated sentence embedding model to determine whether an input is related to previously requested-to-be-forgotten content and dynamically decides whether to respond or refuse. The method enables arbitrary real-time updates to static large language models—a capability not previously achievable—and achieves superior forgetting performance compared to existing techniques while retaining nearly all original knowledge. To the best of our knowledge, this is the first and only framework supporting real-time continual unlearning.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The inability to filter out in advance all potentially problematic data from the pre-training of large language models has given rise to the need for methods for unlearning specific pieces of knowledge after training. Existing techniques overlook the need for continuous and immediate action, causing them to suffer from degraded utility as updates accumulate and protracted exposure of sensitive information. To address these issues, we propose Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge (CURaTE). Our method begins by training a sentence embedding model on a dataset designed to enable the formation of sharp decision boundaries for determining whether a given input prompt corresponds to any stored forget requests. The similarity of a given input to the forget requests is then used to determine whether to answer or return a refusal response. We show that even with such a simple approach, not only does CURaTE achieve more effective forgetting than existing methods, but by avoiding modification of the language model parameters, it also maintains near perfect knowledge preservation over any number of updates and is the only method capable of continual unlearning in real-time.

Problem

• continual unlearning
• real-time unlearning
• large language models
• knowledge preservation
• forgetting

Innovation

• continual unlearning
• real-time unlearning
• knowledge preservation
• sentence embedding
• large language models