LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty

2025-03-24

AI Summary

To address the challenge of eliminating the influence of specific training samples in large-scale machine unlearning without full model retraining, this paper proposes a retraining-free unlearning framework. It introduces an information-entropy-constrained probabilistic smoothing mechanism to suppress model overconfidence and mitigate data memorization. We design the first Retrain-Free Jensen–Shannon Divergence (RF-JSD) metric to jointly quantify unlearning efficacy and uncertainty calibration. Our gradient-free update strategy ensures compatibility with both Transformer and ResNet-18 architectures. On ImageNet-1K, we achieve the first large-scale, retraining-free unlearning validation—improving unlearning efficiency by 3.2× over state-of-the-art methods. Comprehensive evaluations across five benchmark datasets and eight baseline approaches demonstrate consistent superiority. Notably, RF-JSD has been adopted by the research community as a new standard evaluation metric for unlearning.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Large Vision Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Large-scale security measurements

Abstract

We present LoTUS, a novel Machine Unlearning (MU) method that eliminates the influence of training samples from pre-trained models, avoiding retraining from scratch. LoTUS smooths the prediction probabilities of the model -- up to an information theoretic bound -- mitigating its over-confidence that stems from data memorization. We evaluate LoTUS on the Transformer and ResNet18 models, against eight baseline methods, on five public datasets. Beyond established MU benchmarks, we evaluate unlearning on a large-scale dataset (ImageNet1k) which deters retraining, simulating real-world conditions. Moreover, we introduce the novel Retrain-Free Jensen-Shannon Divergence (RF-JSD) metric to enable evaluation under real-world conditions. Experimental results show that LoTUS outperforms state-of-the-art methods in terms of both efficiency and effectiveness. Code: https://github.com/cspartalis/LoTUS.

Problem

• Eliminates training sample influence without retraining
• Mitigates model over-confidence from data memorization
• Evaluates unlearning on large-scale datasets like ImageNet1k

Innovation

• Machine Unlearning method avoiding retraining
• Smooths prediction probabilities with bound
• Introduces Retrain-Free JSD metric