Backdoor Mitigation in Object Detection via Adversarial Fine-Tuning

2026-05-07

AI Summary

This work addresses the challenge of defending object detection models against backdoor attacks in realistic scenarios where only a poisoned model, limited clean data, and no knowledge of the attack target are available. To this end, the authors propose a detection-aware adversarial fine-tuning framework that introduces a novel soft-branch minimization mechanism to jointly handle two common backdoor behaviors—misclassification and target disappearance. The framework employs a dual-objective fine-tuning loss that selectively focuses on predictions most relevant to the backdoor for precise model repair. Compatible with both CNN and Transformer architectures, the method significantly reduces attack success rates across diverse detectors while preserving strong performance on clean samples, substantially outperforming existing backdoor defenses originally designed for classification tasks.

Abstract

Backdoor attacks can implant malicious behaviours into deep models while preserving performance on clean data, posing a serious threat to safety-critical vision systems. Although backdoor mitigation has been studied extensively for image classification, defenses for object detection remain comparatively underdeveloped. Adversarial fine-tuning is a common backdoor mitigation approach in classification, but adapting it to detection is nontrivial as classification-oriented adversarial generation does not match the detection attack space, where attacks may cause object misclassification or disappearance, and standard detection losses can dilute the repair signal across many predictions. We address these challenges through a detection-aware adversarial fine-tuning framework for mitigating object-detection backdoors when the defender has access only to a compromised detector and a small clean dataset, without knowing the attack objective. For adversarial generation that does not require knowledge of the attack objective, we introduce soft-branch minimisation, which uses a soft gate to combine objectives aligned with misclassification and disappearance attacks, together with a detection-aware classification-loss maximisation. For targeted repair, we introduce a dual-objective fine-tuning loss applied to target-matched predictions, concentrating the defensive update on predictions most relevant to the backdoor behaviour. Experiments across CNN- and Transformer-based detectors show that our approach more effectively reduces attack success while preserving true detections, compared with classification-oriented baselines, and maintains competitive clean detection performance.

Problem

• backdoor mitigation
• object detection
• adversarial fine-tuning
• detection-aware defense
• backdoor attacks

Innovation

• adversarial fine-tuning
• backdoor mitigation
• object detection
• soft-branch minimisation
• dual-objective loss