Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information

2026-03-03

AI Summary

This work aims to enhance the effectiveness of “unlearnable examples” in preventing unauthorized deep models from learning from private data. It presents the first analysis of the underlying mechanism from an information-theoretic perspective, revealing that deeper networks exhibit reduced mutual information between clean and perturbed features, and demonstrating that minimizing this mutual information effectively hinders model generalization. Building on this insight, the authors propose a novel method, MI-UE, which generates more potent unlearnable examples by maximizing the cosine similarity among intra-class perturbed features to shrink the conditional covariance. Extensive experiments show that MI-UE consistently outperforms existing approaches and robustly prevents illicit model learning across diverse defense settings.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The volume of freely scraped data on the Internet has driven the tremendous success of deep learning. Along with this comes the growing concern about data privacy and security. Numerous methods for generating unlearnable examples have been proposed to prevent data from being illicitly learned by unauthorized deep models by impeding generalization. However, the existing approaches primarily rely on empirical heuristics, making it challenging to enhance unlearnable examples with solid explanations. In this paper, we analyze and improve unlearnable examples from a novel perspective: mutual information reduction. We demonstrate that effective unlearnable examples always decrease mutual information between clean features and poisoned features, and when the network gets deeper, the unlearnability goes better together with lower mutual information. Further, we prove from a covariance reduction perspective that minimizing the conditional covariance of intra-class poisoned features reduces the mutual information between distributions. Based on the theoretical results, we propose a novel unlearnable method called Mutual Information Unlearnable Examples (MI-UE) that reduces covariance by maximizing the cosine similarity among intra-class features, thus impeding the generalization effectively. Extensive experiments demonstrate that our approach significantly outperforms the previous methods, even under defense mechanisms.

Problem

• unlearnable examples
• data privacy
• mutual information
• deep learning
• generalization

Innovation

• mutual information
• unlearnable examples
• covariance reduction
• feature poisoning
• data privacy