Fast Adversarial Attacks with Gradient Prediction

2026-05-14

AI Summary

This work addresses the computational bottleneck of existing fast adversarial attacks—such as FGSM—which, despite their speed, still rely on backpropagation and thus struggle to meet the throughput demands of large-scale adversarial example generation. To overcome this limitation, we propose the first backpropagation-free, highly efficient adversarial attack method. Drawing upon neural tangent kernel theory, our approach leverages hidden states extracted during forward propagation to predict input gradients via a lightweight linear regression model. The method maintains both theoretical grounding and practical efficacy in finite-width networks, achieving comparable attack performance to FGSM while boosting throughput by 532%, thereby significantly accelerating adversarial sample generation.

Abstract

Generating adversarial examples at scale is a core primitive for robustness evaluation, adversarial training, and red-teaming, yet even "fast" attacks such as FGSM remain throughput-limited by the cost of a backward pass. We introduce a family of attacks that eliminates the backward pass by predicting the input gradient from forward-pass hidden states via a lightweight linear regression. The approach is motivated by a kernel view of neural networks and is exact in the Neural Tangent Kernel regime, while remaining effective for practical finite-width models. Empirically, our methods recover much of FGSM's attack performance while using only a small fraction of the time, corresponding to a $532\%$ increase in throughput. These results suggest gradient prediction as a simple and general route to significantly faster adversarial generation under realistic wall-clock constraints.

Problem

• adversarial attacks
• gradient prediction
• throughput
• robustness evaluation
• backward pass

Innovation

• gradient prediction
• adversarial attacks
• Neural Tangent Kernel
• forward-only attack
• throughput acceleration