Forget by Uncertainty: Orthogonal Entropy Unlearning for Quantized Neural Networks

2026-01-31

AI Summary

This work addresses the challenge of implementing machine unlearning in quantized neural networks that complies with privacy regulations such as GDPR, where unlearning must be distinguished from erroneous memorization and gradient conflicts between forgetting and retention tasks must be resolved. To this end, the authors propose the Orthogonal Entropy Unlearning (OEU) framework, which achieves genuine forgetting by maximizing the predictive entropy of data to be forgotten and employs gradient orthogonal projection to eliminate interference between tasks. The method offers both theoretical guarantees under first-order approximation and strong practical performance, significantly outperforming existing approaches in both the completeness of forgetting and the accuracy of retained tasks.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

The deployment of quantized neural networks on edge devices, combined with privacy regulations like GDPR, creates an urgent need for machine unlearning in quantized models. However, existing methods face critical challenges: they induce forgetting by training models to memorize incorrect labels, conflating forgetting with misremembering, and employ scalar gradient reweighting that cannot resolve directional conflicts between gradients. We propose OEU, a novel Orthogonal Entropy Unlearning framework with two key innovations: 1) Entropy-guided unlearning maximizes prediction uncertainty on forgotten data, achieving genuine forgetting rather than confident misprediction, and 2) Gradient orthogonal projection eliminates interference by projecting forgetting gradients onto the orthogonal complement of retain gradients, providing theoretical guarantees for utility preservation under first-order approximation. Extensive experiments demonstrate that OEU outperforms existing methods in both forgetting effectiveness and retain accuracy.

Problem

• machine unlearning
• quantized neural networks
• privacy regulations
• gradient conflict
• model forgetting

Innovation

• machine unlearning
• quantized neural networks
• entropy maximization
• gradient orthogonal projection
• orthogonal complement