Discard the Dross and Select the Essential: Pre-query Sample Selection for Black-box Membership Inference Attacks

2026-06-29

AI Summary

This work addresses the high query cost and exposure risk of black-box membership inference attacks (MIAs), which stem from indiscriminate querying of candidate samples—many of which lack informative membership signals. To overcome this limitation, the authors propose PSS-MIA, a novel framework that introduces a pre-query sample selection mechanism. By leveraging a reference model, PSS-MIA computes the loss gap—the difference in loss between the target and reference models—for each candidate sample and prioritizes those with larger gaps for querying. This approach is compatible with existing black-box MIA methods and significantly improves query efficiency. Experimental results on CIFAR-10, CIFAR-100, and CINIC-10 demonstrate that, under a 0.1% false positive rate constraint, PSS-MIA reduces query budgets by at least 83.1%, 60.6%, and 80.4%, respectively, substantially outperforming baseline approaches.

Abstract

Black-box membership inference attacks (MIAs) rely on target-model queries to infer whether candidate samples were used for training. However, membership signals are highly non-uniform across samples: some candidate samples support strong member/non-member separability, whereas many others provide little useful signal. Consequently, indiscriminate querying can incur substantial query cost and increase query-induced exposure, with limited marginal benefit for inference. This raises a key question: which candidate samples are worth querying for black-box MIAs? To address this question, we propose PSS-MIA, a pre-query sample selection framework which can be embedded with any existing MIA methods. PSS-MIA proceeds in two stages: it first ranks candidate samples and selects a subset expected to support stronger membership inference, then queries the selected samples and uses the returned outputs for an existing black-box MIA, thereby reducing query cost and query-induced exposure. In the first stage, we propose Loss-Gap Ranking (LGR), which ranks candidate samples by estimating the strength of their membership signal using loss gaps computed from reference models. Experiments on CIFAR-10, CIFAR-100, and CINIC-10 with five representative black-box MIA methods demonstrate that PSS-MIA with LGR consistently outperforms all other compared methods. Moreover, under a 0.1% FPR constraint, PSS-MIA can save at least 83.1%, 60.6%, and 80.4% of the query budget for the three datasets, respectively.

Problem

• membership inference attacks
• black-box
• sample selection
• query cost
• member/non-member separability

Innovation

• pre-query sample selection
• black-box membership inference attack
• Loss-Gap Ranking
• query efficiency
• membership signal