The SkipSponge attack: Sponge weight poisoning of deep neural networks

2024-02-09

ITU Journal on Future and Evolving Technologies

AI Summary

Existing backdoor attacks on pre-trained models rely heavily on large-scale poisoned datasets, limiting practicality and detectability. Method: We propose SkipSponge—a lightweight, weight-oriented sponge attack that directly perturbs model weights using fewer than 1% poisoned samples, without requiring extensive data retraining. Contribution/Results: SkipSponge is the first to achieve gradient-sensitive weight poisoning under extreme sample constraints; it introduces targeted layer-wise bias perturbation and a “sponge-like” computational inflation mechanism to amplify inference latency and energy consumption (up to +13%) in image classification, GANs, and autoencoders—while inducing minimal, stealthy parameter changes. Crucially, it evades mainstream backdoor defenses: experiments show that defense methods not specifically designed for SkipSponge fail completely, yielding high attack success rates and low detectability.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Measurement, analysis, and circumvention of Web censorship
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Sponge attacks aim to increase the energy consumption and computation time of neural networks. In this work, we present a novel sponge attack called SkipSponge. SkipSponge is the first sponge attack that is performed directly on the parameters of a pretrained model using only a few data samples. Our experiments show that SkipSponge can successfully increase the energy consumption of image classification models, GANs, and autoencoders, requiring fewer samples than the state-of-the-art sponge attacks (Sponge Poisoning). We show that poisoning defenses are ineffective if not adjusted specifically for the defense against SkipSponge (i.e., they decrease target layer bias values) and that SkipSponge is more effective on the GANs and the autoencoders than Sponge Poisoning. Additionally, SkipSponge is stealthy as it does not require significant changes to the victim model's parameters. Our experiments indicate that SkipSponge can be performed even when an attacker has access to less than 1% of the entire training dataset and reaches up to 13% energy increase.

Problem

• Increasing energy consumption and computation time of neural networks
• Poisoning pretrained model parameters using minimal data samples
• Bypassing existing defenses while maintaining model stealthiness

Innovation

• Poisoning pretrained model parameters directly
• Requiring only minimal training data samples
• Increasing energy consumption stealthily and effectively