Lower Bounds on the MMSE of Adversarially Inferring Sensitive Features

2025-05-13

AI Summary

This work investigates the inference of sensitive features from noisy, correlated features under limited samples and linear predictive models, and theoretically characterizes the minimum mean squared error (MMSE) lower bound. We propose the first adversarial evaluation framework tailored to sensitive feature inference, deriving the first noise-variance-order-optimal closed-form MMSE lower bound—unifying linear mappings, binary symmetric channels, and conditionally Gaussian feature dependencies. Furthermore, we design a computationally tractable lower bound construction method based on validation-set MSE. Theoretical analysis establishes both order optimality and statistical achievability of the bound. Experiments demonstrate that our framework significantly improves privacy risk quantification efficiency while preserving rigorous theoretical guarantees.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Adversarial Search

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

We propose an adversarial evaluation framework for sensitive feature inference based on minimum mean-squared error (MMSE) estimation with a finite sample size and linear predictive models. Our approach establishes theoretical lower bounds on the true MMSE of inferring sensitive features from noisy observations of other correlated features. These bounds are expressed in terms of the empirical MMSE under a restricted hypothesis class and a non-negative error term. The error term captures both the estimation error due to finite number of samples and the approximation error from using a restricted hypothesis class. For linear predictive models, we derive closed-form bounds, which are order optimal in terms of the noise variance, on the approximation error for several classes of relationships between the sensitive and non-sensitive features, including linear mappings, binary symmetric channels, and class-conditional multi-variate Gaussian distributions. We also present a new lower bound that relies on the MSE computed on a hold-out validation dataset of the MMSE estimator learned on finite-samples and a restricted hypothesis class. Through empirical evaluation, we demonstrate that our framework serves as an effective tool for MMSE-based adversarial evaluation of sensitive feature inference that balances theoretical guarantees with practical efficiency.

Problem

• Establishes lower bounds on MMSE for sensitive feature inference
• Analyzes error from finite samples and restricted hypothesis class
• Derives closed-form bounds for linear predictive models

Innovation

• Adversarial evaluation framework for sensitive feature inference
• Theoretical lower bounds on MMSE with finite samples
• Closed-form bounds for linear predictive models