Range Membership Inference Attacks

2024-08-09

arXiv.org

AI Summary

Existing membership inference attacks (MIAs) only detect exact matches with training data, overlooking privacy leakage risks arising from semantically similar or distributionally proximal samples. This paper introduces Range-based Membership Inference Attack (RaMIA), the first framework to formally define and address the novel problem of determining whether a model was trained on data within a given semantic range—such as visually similar images or samples drawn from the same underlying distribution. RaMIA establishes a statistically rigorous, hypothesis-testing-based framework that requires no access to training data or labels, and is applicable to tabular, image, and language models. Extensive experiments demonstrate that RaMIA significantly improves detection rates of privacy leakage across diverse data modalities, outperforming conventional MIAs in both accuracy and robustness. By incorporating semantic and distributional awareness, RaMIA provides a more comprehensive and principled benchmark for assessing privacy risks in machine learning models.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Machine learning models can leak private information about their training data. The standard methods to measure this privacy risk, based on membership inference attacks (MIAs), only check if a given data point extit{exactly} matches a training point, neglecting the potential of similar or partially overlapping memorized data revealing the same private information. To address this issue, we introduce the class of range membership inference attacks (RaMIAs), testing if the model was trained on any data in a specified range (defined based on the semantics of privacy). We formulate the RaMIAs game and design a principled statistical test for its composite hypotheses. We show that RaMIAs can capture privacy loss more accurately and comprehensively than MIAs on various types of data, such as tabular, image, and language. RaMIA paves the way for more comprehensive and meaningful privacy auditing of machine learning algorithms.

Problem

• Measure privacy risk in machine learning models
• Introduce range membership inference attacks (RaMIAs)
• Improve privacy auditing accuracy and comprehensiveness

Innovation

• Introduces range membership inference attacks (RaMIAs)
• Tests if model trained on data within specified range
• Provides more comprehensive privacy auditing for ML