Data-Free Model-Related Attacks: Unleashing the Potential of Generative AI

2025-01-28

AI Summary

This work identifies a novel security threat to deep learning models posed by generative AI—specifically diffusion models and large language models—focusing on three black-box, data-agnostic attacks: model extraction, membership inference, and model inversion. We propose the first unified attack framework that requires neither training data nor model parameters: it leverages zero-shot prompting to synthesize surrogate data, then employs adversarial feature distillation and gradient approximation to estimate gradients. Our method overcomes traditional constraints of white-box access or data dependency, achieving performance comparable to white-box baselines—85%+ model extraction accuracy and 0.92 AUC for membership inference—on both image and text models. This is the first systematic demonstration of practical, fully black-box, data-free multi-task model attacks empowered by generative AI, establishing a new research direction in AI security.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Generative AI technology has become increasingly integrated into our daily lives, offering powerful capabilities to enhance productivity. However, these same capabilities can be exploited by adversaries for malicious purposes. While existing research on adversarial applications of generative AI predominantly focuses on cyberattacks, less attention has been given to attacks targeting deep learning models. In this paper, we introduce the use of generative AI for facilitating model-related attacks, including model extraction, membership inference, and model inversion. Our study reveals that adversaries can launch a variety of model-related attacks against both image and text models in a data-free and black-box manner, achieving comparable performance to baseline methods that have access to the target models' training data and parameters in a white-box manner. This research serves as an important early warning to the community about the potential risks associated with generative AI-powered attacks on deep learning models.

Problem

• Generative AI
• Deep Learning Attacks
• Data Inference

Innovation

• Generative AI
• Model Attack
• AI Security Risk