Reducing information dependency does not cause training data privacy. Adversarially non-robust features do

2026-07-14

AI Summary

This work challenges the prevailing view that information dependence inherently leads to privacy leakage under model inversion attacks (MIAs), demonstrating instead that the root cause lies in adversarially non-robust features rather than memorization. Through theoretical analysis and empirical validation, the study establishes the first causal link between adversarially non-robust features and MIA vulnerability, revealing a novel trade-off between privacy and robustness. To address this, the authors propose Anti Adversarial Training (AT-AT), a mechanism that actively exploits such features to enhance privacy protection. Under extreme training conditions—such as 97% pixel masking—and across multiple evaluation metrics including HSIC and MIA success rates, AT-AT significantly outperforms existing defenses, achieving higher model accuracy while strengthening privacy guarantees. Notably, the work also shows that models satisfying strong information-theoretic privacy bounds can still be severely reconstructed.

Abstract

In this paper, we challenge the prevailing view that information dependency (including rote memorization) drives training data exposure to image reconstruction attacks. We show that extensive exposure can persist without rote memorization and is instead caused by a tunable connection to adversarial robustness. We begin by presenting three surprising results: (1) recent defenses that inhibit reconstruction by Model Inversion Attacks (MIAs), which evaluate leakage under an idealized attacker, do not reduce standard measures of information dependency (HSIC); (2) models that maximally memorize their training datasets remain robust to MIA reconstruction; and (3) models trained without seeing 97% of the training pixels, where recent information-theoretic bounds give arbitrarily strong privacy guarantees under standard assumptions, can still be devastatingly reconstructed by MIA. To explain these findings, we provide causal evidence that privacy under MIA arises from what the adversarial examples literature calls ``non-robust'' features (generalizable but imperceptible and unstable features). We further show that recent MIA defenses obtain their privacy improvements by unintentionally shifting models toward such features. To establish this causal relationship, we introduce Anti Adversarial Training (AT-AT), a training regime that intentionally learns non-robust features to obtain both superior reconstruction defense and higher accuracy than state-of-the-art defenses. Our results revise the prevailing understanding of training data exposure and reveal a new privacy-robustness tradeoff.

Problem

• training data privacy
• model inversion attacks
• information dependency
• adversarial robustness
• non-robust features

Innovation

• non-robust features
• model inversion attacks
• adversarial robustness
• Anti Adversarial Training
• training data privacy