Rethinking Machine Unlearning: Models Designed to Forget via Key Deletion

2026-03-16

AI Summary

This work proposes a novel “forgetting-by-design” paradigm that embeds forgetting capability directly into the training phase, eliminating the need for post-hoc processing, model retraining, or access to original training data. Unlike existing machine unlearning methods that rely on retrospective interventions and full data availability, the proposed approach decouples instance-specific memory from model weights, enabling zero-shot, instantaneous forgetting through the deletion of instance identifier keys. Built upon a memory-augmented Transformer architecture (MUNKEY), the framework supports targeted forgetting operations while preserving predictive performance. Experiments across natural image, fine-grained recognition, and medical datasets demonstrate that the method consistently outperforms state-of-the-art post-hoc unlearning techniques, achieving deployment-ready forgetting speeds without compromising accuracy.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Computer Vision: Large Vision Models
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Machine unlearning is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training samples. Despite this, most existing methods tackle the problem purely from a post-hoc perspective. They attempt to erase the influence of targeted training samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We propose \textit{unlearning by design}, a novel paradigm in which models are directly trained to support forgetting as an inherent capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling direct zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.

Problem

• machine unlearning
• privacy regulations
• data deletion
• model forgetting
• training data removal

Innovation

• machine unlearning
• unlearning by design
• key deletion
• memory-augmented transformer
• zero-shot forgetting