A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning

2026-07-03

AI Summary

This work addresses the insufficient adversarial robustness in unsupervised domain adaptation caused by noisy pseudo-labels and source–target distribution shifts. To tackle this, the authors propose a two-stage SFT+RL framework built upon the CLIP vision encoder. In the first stage, they perform PGD-based adversarial fine-tuning of the linear classifier while partially unfreezing the projection layer. The second stage introduces a confidence-guided progressive pseudo-labeling strategy that employs a dynamically decaying threshold to select high-confidence target samples, which are then used to construct a hybrid dataset for reinforcement learning–driven adversarial training. This approach preserves CLIP’s semantic priors while substantially enhancing cross-domain robustness and accuracy, achieving an average improvement of 10.2% in clean accuracy and 15.8% in adversarial robustness over state-of-the-art methods on OfficeHome, PACS, and VisDA benchmarks.

Abstract

Adversarial robustness in Unsupervised Domain Adaptation (UDA) remains a significant challenge due to noisy pseudo labels and inherent distributional shifts between the clean source and adversarially perturbed target domains. Existing approaches often fail to achieve an optimal trade-off between robustness and accuracy, as pseudo-labels generated by domain-adapted models tend to introduce classification errors under adversarial attacks. In this work, we propose \textbf{SFT+RL}, a two-stage robust UDA framework that integrates Supervised Fine Tuning (SFT) and Reinforcement Learning (RL) on top of CLIP's pre-trained visual encoder. In the SFT stage, we adversarially fine-tune a linear classifier using PGD-based perturbations over the labelled source domain while partially unfreezing CLIP's projection layer. It allows adaptation to adversarial noise while preserving CLIP's rich semantic priors. We introduce a confidence-guided pseudo-labeling strategy in the RL stage to annotate unlabeled target samples progressively. Pseudo labels are filtered using a decaying confidence threshold to balance quality and coverage, and the model is trained on a composite dataset formed by combining clean source samples with high-confidence target samples. Adversarial training is applied to mixed batches of clean and adversarial examples to enhance cross-domain robustness. Comprehensive evaluations on three benchmark datasets OfficeHome~\cite{tomm-ude}, PACS~\cite{pacs}, and VisDA~\cite{visda} demonstrate the effectiveness of our approach. Notably, \textbf{SFT+RL} achieves average improvements of \textbf{10.2\%} in clean accuracy and \textbf{15.8\%} in adversarial robustness across all three datasets, outperforming existing state-of-the-art methods.

Problem

• Unsupervised Domain Adaptation
• Adversarial Robustness
• Pseudo Labels
• Distributional Shift
• Domain Adaptation

Innovation

• Unsupervised Domain Adaptation
• Adversarial Robustness
• Supervised Fine-Tuning
• Reinforcement Learning
• CLIP