IMU: Influence-guided Machine Unlearning

2025-08-03

AI Summary

This work addresses the challenge of machine unlearning without access to either retained data or the original training set. We propose an efficient selective unlearning method that relies solely on the forget set. Our approach features: (1) a dynamic forgetting intensity allocation mechanism grounded in per-sample influence estimation; and (2) a gradient-ascent-driven adaptive unlearning algorithm that directly optimizes model parameters without retained data. By eliminating the need for auxiliary data generation or full retraining, our method achieves superior performance over existing retain-data-free unlearning baselines on both vision and language tasks. It demonstrates enhanced scalability, numerical stability, and privacy preservation—particularly in large-scale forgetting scenarios—while maintaining model utility on retained examples.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent studies have shown that deep learning models are vulnerable to attacks and tend to memorize training data points, raising significant concerns about privacy leakage. This motivates the development of machine unlearning (MU), i.e., a paradigm that enables models to selectively forget specific data points upon request. However, most existing MU algorithms require partial or full fine-tuning on the retain set. This necessitates continued access to the original training data, which is often impractical due to privacy concerns and storage constraints. A few retain-data-free MU methods have been proposed, but some rely on access to auxiliary data and precomputed statistics of the retain set, while others scale poorly when forgetting larger portions of data. In this paper, we propose Influence-guided Machine Unlearning (IMU), a simple yet effective method that conducts MU using only the forget set. Specifically, IMU employs gradient ascent and innovatively introduces dynamic allocation of unlearning intensities across different data points based on their influences. This adaptive strategy significantly enhances unlearning effectiveness while maintaining model utility. Results across vision and language tasks demonstrate that IMU consistently outperforms existing retain-data-free MU methods.

Problem

• Selectively forget specific data points in models
• Avoid needing original training data for unlearning
• Improve unlearning effectiveness without retain set

Innovation

• Uses only forget set for unlearning
• Dynamic allocation of unlearning intensities
• Gradient ascent based influence guidance