Q-FAKER: Query-free Hard Black-box Attack via Controlled Generation

2025-04-18

AI Summary

Existing hard black-box adversarial attacks rely heavily on extensive model queries or access to target model outputs, resulting in high computational overhead and limited practicality. This paper introduces the first fully query-free hard black-box attack framework: it requires no access to the target model, performs zero inference queries, and instead leverages controllable text generation to drive proxy-model distillation and gradient-free adversarial optimization—yielding highly transferable and natural adversarial examples. The core innovation lies in establishing a target-agnostic, low-cost controllable adversarial generation paradigm. Evaluated across eight benchmark datasets, our method significantly improves both cross-model attack success rates and textual naturalness, while reducing attack cost by one to two orders of magnitude. To our knowledge, this is the first approach enabling practical robustness evaluation of closed-source language models.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Many adversarial attack approaches are proposed to verify the vulnerability of language models. However, they require numerous queries and the information on the target model. Even black-box attack methods also require the target model's output information. They are not applicable in real-world scenarios, as in hard black-box settings where the target model is closed and inaccessible. Even the recently proposed hard black-box attacks still require many queries and demand extremely high costs for training adversarial generators. To address these challenges, we propose Q-faker (Query-free Hard Black-box Attacker), a novel and efficient method that generates adversarial examples without accessing the target model. To avoid accessing the target model, we use a surrogate model instead. The surrogate model generates adversarial sentences for a target-agnostic attack. During this process, we leverage controlled generation techniques. We evaluate our proposed method on eight datasets. Experimental results demonstrate our method's effectiveness including high transferability and the high quality of the generated adversarial examples, and prove its practical in hard black-box settings.

Problem

• Develops query-free attack for inaccessible language models
• Uses surrogate model to generate adversarial sentences
• Achieves high transferability without target model access

Innovation

• Uses surrogate model for target-agnostic attack
• Leverages controlled generation techniques
• Generates adversarial examples without queries