TsallisPGD: Adaptive Gradient Weighting for Adversarial Attacks on Semantic Segmentation

2026-05-05

AI Summary

This work addresses the inefficiency of conventional cross-entropy loss in adversarial attacks on semantic segmentation, which tends to overemphasize already misclassified regions, leading to suboptimal optimization and an overestimation of model robustness. To overcome this limitation, the authors propose TsallisPGD, the first method to integrate Tsallis cross-entropy into semantic segmentation attacks. By introducing a tunable parameter \( q \), TsallisPGD adaptively reweights pixel-wise gradients and incorporates a dynamic scheduling strategy that sweeps across \( q \) values during optimization to enhance attack effectiveness. Extensive experiments demonstrate that TsallisPGD consistently outperforms existing approaches—including CEPGD and SegPGD—on Cityscapes, Pascal VOC, and ADE20K, achieving state-of-the-art average performance in degrading both standard and robust models’ accuracy and mIoU, while exhibiting superior generalization across datasets, architectures, and perturbation budgets.

Abstract

Attacking semantic segmentation models is significantly harder than image classification models because an attacker must flip thousands of pixel predictions simultaneously. Standard pixel-wise cross-entropy (CE) is ill-suited to this setting: it tends to overemphasize already-misclassified pixels, which slows optimization and overstates model robustness. To address these issues, we introduce TsallisPGD, an adversarial attack built on the Tsallis cross-entropy, a generalization of CE parameterized by $q$, which adaptively reshapes the gradient landscape by controlling gradient concentration across pixels. By varying $q$, we steer the attack toward pixels at different confidence levels. We first show that no single fixed-$q$ is universally optimal, as its effectiveness depends on the dataset, model architecture, and perturbation budget. Motivated by this, we propose a dynamic $q$-schedule that sweeps $q$ during optimization. Extensive experiments on Cityscapes, Pascal VOC, and ADE20K show that TsallisPGD, using a single validation-selected schedule, achieves the best average attack rank across all evaluated settings and improves over CEPGD, SegPGD, CosPGD, JSPGD, and MaskedPGD in reducing accuracy and mIoU on both standard and robust models.

Problem

• adversarial attacks
• semantic segmentation
• cross-entropy
• gradient weighting
• robustness

Innovation

• Tsallis cross-entropy
• adaptive gradient weighting
• dynamic q-schedule
• adversarial attack
• semantic segmentation