🤖 AI Summary
This work addresses the privacy and intellectual property risks posed by the reliance of vision–language–action (VLA) models on costly, sensitive training data by presenting the first systematic study of membership inference attacks (MIAs) against such models. The authors propose VLALeaks, a novel attack method that leverages differences in attention mechanisms to extract membership signals in two stages and construct an effective attack model capable of determining whether a given sample was part of the training set. Extensive experiments across multiple VLA benchmarks demonstrate that VLALeaks substantially outperforms existing approaches, achieving state-of-the-art performance in terms of AUC and TPR@1%FPR. This study not only exposes critical privacy vulnerabilities in deployed VLA systems but also fills a significant gap in the literature on MIAs for multimodal embodied AI models.
📝 Abstract
Vision-Language-Action (VLA) models enable end-to-end robot control and have garnered widespread attention. However, the memorization of training data inherent to VLA, coupled with the high cost of robotic data acquisition, raises serious concerns regarding data privacy leakage and intellectual property infringement. Membership inference attacks (MIAs) aim to determine whether a given sample belongs to the training set. While representing a significant privacy threat, this attack remains underexplored in the context of VLA models. To bridge this gap, we propose VLALeaks, which is based on attention discrepancies in VLA models. We reveal, for the first time, the privacy vulnerabilities of VLA models. Specifically, it comprises a two-stage process: (1) membership feature extraction, and (2) attack model construction. Experimental results across multiple VLA benchmarks demonstrate that VLALeaks readily reveals membership information and achieves optimal attack AUC and TPR@1\%FPR, highlighting the privacy vulnerabilities in current VLA model deployments. Our work is the first systematic study of MIAs on VLA models, aiming to provide insights for secure and trustworthy VLA models.