Designing to Forget: Deep Semi-parametric Models for Unlearning

2026-03-24

AI Summary

This work addresses the inefficiency of existing machine unlearning methods, which typically require full model retraining to remove specific training samples. To overcome this limitation, the authors propose a deep semi-parametric model (SPM) that, for the first time, integrates a semi-parametric architecture into machine unlearning. The approach explicitly encodes each training sample through a sample-level information fusion module and enables direct removal of designated samples at test time via a non-parametric mechanism—without altering model parameters or retraining. Evaluated on ImageNet classification, SPM achieves performance comparable to fully parametric models while accelerating unlearning by over 10× and reducing the prediction gap against the retraining baseline by 11%, effectively balancing high unlearning efficiency with strong task accuracy.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Recent advances in machine unlearning have focused on developing algorithms to remove specific training samples from a trained model. In contrast, we observe that not all models are equally easy to unlearn. Hence, we introduce a family of deep semi-parametric models (SPMs) that exhibit non-parametric behavior during unlearning. SPMs use a fusion module that aggregates information from each training sample, enabling explicit test-time deletion of selected samples without altering model parameters. Empirically, we demonstrate that SPMs achieve competitive task performance to parametric models in image classification and generation, while being significantly more efficient for unlearning. Notably, on ImageNet classification, SPMs reduce the prediction gap relative to a retrained (oracle) baseline by $11\%$ and achieve over $10\times$ faster unlearning compared to existing approaches on parametric models. The code is available at https://github.com/amberyzheng/spm_unlearning.

Problem

• machine unlearning
• deep semi-parametric models
• data deletion
• model forgetting
• training sample removal

Innovation

• machine unlearning
• semi-parametric models
• sample deletion
• deep learning
• efficient unlearning