Dithering Defense: Adversarial Robustness of Vision Foundation Models via Multi-Level Floyd-Steinberg Dithering

2026-05-21

AI Summary

Vision foundation models are vulnerable to adversarial attacks, posing a single point of failure for downstream tasks and necessitating lightweight, general-purpose defenses. This work proposes a model-agnostic input preprocessing approach that extends multi-level Floyd–Steinberg error diffusion dithering—originally designed for binary quantization—to intermediate quantization levels, combined with image blurring as a postprocessing step. This strategy effectively disrupts adversarial perturbations while preserving semantic content. Evaluated across six vision tasks and two foundation models, the method consistently outperforms existing defenses, including diffusion-based denoising, under strong attacks such as PGD, MI-FGSM, and SIA, achieving a superior trade-off between robustness and clean-sample accuracy. Its resilience against adaptive attacks is further validated using a straight-through estimator.

Abstract

Vision foundation models are widely used as frozen backbones across many downstream tasks, making them a single point of failure under adversarial attack. We study multi-level Floyd-Steinberg error-diffusion dithering as a lightweight, model-agnostic input transformation that disrupts adversarial perturbations while preserving semantic content. Unlike prior work, which was limited to binary dithering, grayscale CIFAR-10, and a single small model trained from scratch, we evaluate across six tasks (classification, segmentation, depth estimation, retrieval, captioning, visual question answering), two model families (DINOv2, PaliGemma), and three attacks of increasing strength (PGD, MI-FGSM, SIA), as well as an adaptive attacker using a straight-through estimator. Our results show that Floyd-Steinberg dithering at intermediate quantization levels, especially when combined with post-processing blur, exceeds or matches all tested baselines, including diffusion-based denoising, with substantially less degradation on clean inputs.

Problem

• adversarial robustness
• vision foundation models
• dithering
• input transformation
• adversarial attacks

Innovation

• Floyd-Steinberg dithering
• adversarial robustness
• vision foundation models
• model-agnostic defense
• multi-level quantization