Spurious Privacy Leakage in Neural Networks

2025-05-26

AI Summary

This work identifies “spurious privacy leakage” in neural networks: models exhibit stronger memorization of spurious correlation groups, leading to significantly higher membership inference attack success rates—up to 37% higher than for non-spurious groups—especially on simple tasks. We find that existing spurious-correlation mitigation methods (e.g., IRM, GroupDRO) fail to alleviate this privacy inequality. To address this, we propose a novel memory-centric perspective on privacy robustness. Crucially, we conduct the first systematic investigation of architectural impact, revealing that Vision Transformers (ViTs) exhibit greater privacy vulnerability than ResNets under spurious data—a finding that challenges prevailing assumptions. Our methodology includes comprehensive privacy attack evaluations on standard benchmarks (Waterbirds, CelebA), comparative analysis of spurious-robust training strategies, and cross-architecture privacy assessment. Results demonstrate that spurious correlations induce non-uniform privacy risks across feature groups and architectures, exposing critical limitations in current privacy-preserving learning paradigms.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Neural networks are vulnerable to privacy attacks aimed at stealing sensitive data. The risks can be amplified in a real-world scenario, particularly when models are trained on limited and biased data. In this work, we investigate the impact of spurious correlation bias on privacy vulnerability. We introduce emph{spurious privacy leakage}, a phenomenon where spurious groups are significantly more vulnerable to privacy attacks than non-spurious groups. We further show that group privacy disparity increases in tasks with simpler objectives (e.g. fewer classes) due to the persistence of spurious features. Surprisingly, we find that reducing spurious correlation using spurious robust methods does not mitigate spurious privacy leakage. This leads us to introduce a perspective on privacy disparity based on memorization, where mitigating spurious correlation does not mitigate the memorization of spurious data, and therefore, neither the privacy level. Lastly, we compare the privacy of different model architectures trained with spurious data, demonstrating that, contrary to prior works, architectural choice can affect privacy outcomes.

Problem

• Investigates spurious correlation bias on privacy vulnerability
• Introduces spurious privacy leakage in vulnerable groups
• Examines memorization impact on privacy disparity mitigation

Innovation

• Investigates spurious correlation bias on privacy
• Introduces spurious privacy leakage phenomenon
• Compares privacy across different model architectures