Membership Inference Attacks against Large Audio Language Models

2026-03-30

AI Summary

This study addresses the vulnerability of existing membership inference attacks (MIAs) on large audio language models (LALMs) to distributional shifts caused by non-semantic acoustic features—such as speaker voiceprints—between training and test data, which can lead to spuriously inflated attack performance. To mitigate this, the authors propose a multimodal blind baseline that integrates textual, spectral, and prosodic features to identify and eliminate such spurious correlations. By constructing distribution-matched datasets, they establish a more reliable auditing benchmark for LALMs. The work further reveals, for the first time, that LALM memorization exhibits cross-modal characteristics: it manifests only when a speaker’s voiceprint is bound to specific textual content. Experiments demonstrate near-perfect train/test separability (AUC ≈ 1.0) in common speech datasets and a strong correlation (r > 0.7) between standard MIA scores and blind acoustic artifacts, validating the proposed approach.

Abstract

We present the first systematic Membership Inference Attack (MIA) evaluation of Large Audio Language Models (LALMs). As audio encodes non-semantic information, it induces severe train and test distribution shifts and can lead to spurious MIA performance. Using a multi-modal blind baseline based on textual, spectral, and prosodic features, we demonstrate that common speech datasets exhibit near-perfect train/test separability (AUC approximately 1.0) even without model inference, and the standard MIA scores strongly correlate with these blind acoustic artifacts (correlation greater than 0.7). Using this blind baseline, we identify that distribution-matched datasets enable reliable MIA evaluation without distribution shift confounds. We benchmark multiple MIA methods and conduct modality disentanglement experiments on these datasets. The results reveal that LALM memorization is cross-modal, arising only from binding a speaker's vocal identity with its text. These findings establish a principled standard for auditing LALMs beyond spurious correlations.

Problem

• Membership Inference Attack
• Large Audio Language Models
• distribution shift
• train/test separability
• spurious correlation

Innovation

• Membership Inference Attack
• Large Audio Language Models
• Distribution Shift
• Cross-modal Memorization
• Acoustic Artifacts