Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

2026-08-17

AI Summary

This study addresses the high computational costs and limited scalability of existing machine unlearning methods by proposing the L2UL framework. Inspired by Learning to Optimize, this work introduces a novel learning-based, model-agnostic unlearning paradigm that automatically learns unrolling behaviors from a distributional perspective, replacing traditional manual design. Experimental results demonstrate that L2UL achieves accuracy comparable to full retraining on architectures such as ResNet while delivering substantial efficiency gains. Furthermore, the framework effectively validates its scalability on large-scale datasets. Collectively, these findings establish L2UL as a promising new avenue for efficient machine unlearning, overcoming critical bottlenecks in current methodologies.

Abstract

Various machine unlearning techniques have been developed in response to privacy legislation requirements, enabling individuals to exercise their legal right to have their data $D_f$ removed from a machine learning model. This process is typically accomplished via the use of an unlearning function denoted as $U$. Existing methods focus on designing an intricate $U$ to unlearn $D_f \subset D$ from a previous model $A(D)$, so that the unlearned model performs as closely as possible to the retrained model $A(D \setminus D_f)$. However, these methods often suffer from high computational costs when dealing with massive training data, as the complex structures of $U$ become a bottleneck even for models with fewer parameters. Inspired by Learning to Optimize, we introduce the first learning-based model-agnostic approach, Learning-to-UnLearn (L2UL). Our core insight is to shift from manually designing $U$ to learning the unlearning behaviors from a distribution perspective, thereby acquiring a simple and efficient $U$ via learning. Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining while exhibiting impressive efficiency, particularly in data-intensive scenarios. Furthermore, we validate the performance and scalability of our method on larger models ResNet.

Problem

• Machine Unlearning
• Computational Cost
• Efficiency
• Data-intensive Scenarios

Innovation

• Machine Unlearning
• Learning-to-UnLearn
• Model-agnostic
• Learning to Optimize
• Computational Efficiency