CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training

2026-04-14

AI Summary

This work challenges the common assumption that subset training is inherently safer by revealing that its data selection process can introduce novel privacy vulnerabilities. The authors propose CoLA, a unified framework that extends privacy risk analysis beyond model training to encompass the entire data-to-model supply chain. They introduce a new threat paradigm—Selection Participation Membership Inference Attack (SP-MIA)—which, alongside traditional Training Membership Inference Attacks (TM-MIA), defines two distinct attack scenarios. CoLA leverages side-channel metadata and black-box model outputs to develop Subset-aware and Black-box attack strategies applicable to both vision and language models. Empirical results demonstrate that existing threat models significantly underestimate the privacy risks of subset training, whereas CoLA effectively infers both training membership and selection participation, exposing ecosystem-level privacy threats.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Training models on a carefully chosen portion of data rather than the full dataset is now a standard preprocess for modern ML. From vision coreset selection to large-scale filtering in language models, it enables scalability with minimal utility loss. A common intuition is that training on fewer samples should also reduce privacy risks. In this paper, we challenge this assumption. We show that subset training is not privacy free: the very choices of which data are included or excluded can introduce new privacy surface and leak more sensitive information. Such information can be captured by adversaries either through side-channel metadata from the subset selection process or via the outputs of the target model. To systematically study this phenomenon, we propose CoLA (Choice Leakage Attack), a unified framework for analyzing privacy leakage in subset selection. In CoLA, depending on the adversary's knowledge of the side-channel information, we define two practical attack scenarios: Subset-aware Side-channel Attacks and Black-box Attacks. Under both scenarios, we investigate two privacy surfaces unique to subset training: (1) Training-membership MIA (TM-MIA), which concerns only the privacy of training data membership, and (2) Selection-participation MIA (SP-MIA), which concerns the privacy of all samples that participated in the subset selection process. Notably, SP-MIA enlarges the notion of membership from model training to the entire data-model supply chain. Experiments on vision and language models show that existing threat models underestimate subset-training privacy risks: the expanded privacy surface leaks both training and selection membership, extending risks from individual models to the broader ML ecosystem.

Problem

• privacy leakage
• subset training
• membership inference
• side-channel attack
• data selection

Innovation

• Choice Leakage
• Subset Training
• Membership Inference Attack
• Privacy Risk
• CoLA