MCU: Improving Machine Unlearning through Mode Connectivity

2025-05-16

AI Summary

In machine unlearning, incomplete privacy erasure and performance degradation arise from weight entanglement. Method: This paper proposes a nonlinear unlearning path construction method grounded in mode connectivity—introducing this concept to machine unlearning for the first time to reveal a continuous spectrum of unlearned models. It designs a learnable parameter mask and an adaptive regularization mechanism to automatically modulate unlearning intensity, eliminating manual hyperparameter tuning. Sparse masking constrains updates to critical parameters, while task arithmetic enhances generalization stability. Results: On image classification benchmarks, the method significantly improves unlearning quality (ΔFID reduced by 32.1%) and model utility retention (Top-1 accuracy drops only 0.8%), reduces computational overhead by 41%, and serves as a plug-and-play enhancement applicable to diverse baseline unlearning algorithms.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Learning to Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Machine Unlearning (MU) aims to remove the information of specific training data from a trained model, ensuring compliance with privacy regulations and user requests. While one line of existing MU methods relies on linear parameter updates via task arithmetic, they suffer from weight entanglement. In this work, we propose a novel MU framework called Mode Connectivity Unlearning (MCU) that leverages mode connectivity to find an unlearning pathway in a nonlinear manner. To further enhance performance and efficiency, we introduce a parameter mask strategy that not only improves unlearning effectiveness but also reduces computational overhead. Moreover, we propose an adaptive adjustment strategy for our unlearning penalty coefficient to adaptively balance forgetting quality and predictive performance during training, eliminating the need for empirical hyperparameter tuning. Unlike traditional MU methods that identify only a single unlearning model, MCU uncovers a spectrum of unlearning models along the pathway. Overall, MCU serves as a plug-and-play framework that seamlessly integrates with any existing MU methods, consistently improving unlearning efficacy. Extensive experiments on the image classification task demonstrate that MCU achieves superior performance.

Problem

• Enhance machine unlearning via nonlinear mode connectivity
• Improve unlearning efficiency with parameter masking
• Balance forgetting and model performance adaptively

Innovation

• Leverages mode connectivity for nonlinear unlearning pathway
• Introduces parameter mask to boost efficiency and effectiveness
• Adaptively adjusts penalty coefficient for balanced performance