One Framework for All: Cross-Modal Membership Inference for Generative Models

2026-07-05

AI Summary

Existing membership inference attacks are typically tailored to specific modalities, lacking cross-modal generalizability. This work proposes the first unified cross-modal membership inference framework that leverages the modality-agnostic property that a generative model’s output distribution approximates the distribution of its training data. By modeling generated samples and non-member samples in a shared embedding space and employing likelihood ratio testing for inference, the method operates under black-box settings and supports both partial-knowledge and zero-knowledge threat models. It is applicable across diverse generative architectures, including text-to-text, text-to-image, and image-to-text models. Experiments demonstrate that the proposed approach significantly outperforms state-of-the-art modality-specific methods on both fine-tuned and pretraining data, thereby overcoming the limitations of single-modality attacks and enhancing the generality and practicality of membership inference.

Abstract

Large generative models across text-to-text, text-to-image, and image-to-text modalities have been shown to pose significant privacy risks. One fundamental threat is membership inference attacks (MIA), which aim to determine whether a given data point was used in a model's training set. Although prior work has investigated MIAs against these three classes of generative models, existing approaches treat them in isolation and are not cross-applicable, thereby limiting their real-world utility. To address this limitation, we present the first comprehensive study of a unified membership inference framework that applies across text-to-text, text-to-image, and image-to-text modalities. Our approach is grounded in a key modality-agnostic observation: the output distribution of a generative model can approximate its training data distribution. Leveraging this property, we model the distributions of model-generated outputs and auxiliary non-member samples in a shared embedding space, and perform membership inference via likelihood ratio testing. We conduct extensive experiments in a strict black-box setting under both partial-knowledge and zero-knowledge threat models, and evaluate membership inference against both fine-tuning and pre-training data. Experimental results demonstrate our approach's superior performance in comparison to existing state-of-the-art methods, which are typically optimized for a single model class.

Problem

• membership inference
• generative models
• cross-modal
• privacy risk
• unified framework

Innovation

• membership inference
• cross-modal
• generative models
• likelihood ratio test
• black-box attack