IDATA: Scalable Invertible Diffusion for Unrestricted Adversarial Transfer Attack

2026-08-09

AI Summary

Existing diffusion-based unrestricted adversarial transfer attacks struggle to balance scalability and effectiveness due to high memory overhead and insensitivity to perturbation frequency in latent spaces. This work proposes IDATA, a novel framework that uniquely integrates invertible computation with wavelet frequency-domain constraints. Specifically, it introduces an Invertible Diffusion Module (IDM) enabling constant-memory backpropagation and leverages Discrete Wavelet Transform (DWT) to confine perturbations to the semantically stable low-frequency subspace. Evaluated across multiple benchmarks, IDATA substantially outperforms current state-of-the-art methods, achieving simultaneous improvements in attack success rate, memory efficiency, and visual imperceptibility.

Abstract

Unrestricted adversarial transfer attacks are important for evaluating the black-box robustness of deep visual models. Diffusion-based attacks have shown promising transferability and visual imperceptibility by optimizing adversarial perturbations along denoising trajectories in latent space. However, existing methods are limited by two challenges: memory-intensive multistep backpropagation and frequency-agnostic perturbation over intermediate latents. To address these issues, we propose IDATA, a memory-efficient diffusion framework for unrestricted adversarial transfer attack. IDATA consists of two key components: an Invertible Diffusion Module (IDM) and a Low-Frequency Constraint Module (LFCM). Specifically, IDM reformulates adversarial optimization over diffusion trajectories as an invertible process, enabling constant-memory backpropagation through on-demand reconstruction of intermediate states instead of storing the full denoising chain. Moreover, LFCM leverages Discrete Wavelet Transform (DWT) to decompose latent variables into low- and high-frequency components, restricting perturbations to semantically stable low-frequency subspaces, thereby improving transferability while preserving visual imperceptibility. Extensive experiments on multiple benchmarks and diverse model architectures demonstrate that IDATA consistently outperforms state-of-the-art baselines in attack success rate, memory efficiency, and visual imperceptibility. These results suggest that IDATA is a promising tool for black-box robustness evaluation of deep visual models. Code is available at https://github.com/colourful-pan/IDATA.

Problem

• unrestricted adversarial transfer attack
• diffusion-based attack
• memory-intensive backpropagation
• frequency-agnostic perturbation
• black-box robustness

Innovation

• Invertible Diffusion
• Low-Frequency Constraint
• Adversarial Transfer Attack
• Memory-Efficient Backpropagation
• Discrete Wavelet Transform