Shuffle PatchMix Augmentation with Confidence-Margin Weighted Pseudo-Labels for Enhanced Source-Free Domain Adaptation

2025-05-30

AI Summary

This paper addresses source-free domain adaptation (SFDA), where source data is unavailable during adaptation. To tackle challenges such as high noise in target-domain pseudo-labels and overfitting on small target samples, we propose Shuffle PatchMix (SPM)—a patch-level cross-sample shuffling and mixing augmentation—and a confidence-boundary-driven dynamic pseudo-label reweighting mechanism. SPM enhances feature robustness by randomly permuting and blending image patches across target samples, while the reweighting mechanism adaptively suppresses low-confidence pseudo-labels to mitigate label noise. Jointly optimizing unsupervised target-domain training, our approach achieves state-of-the-art performance on three major benchmarks: PACS (improvements of +7.3% and +7.2% under single- and multi-target settings), DomainNet-126 (+2.8%), and VisDA-C (+0.7%). These results demonstrate significant gains in generalization capability and robustness for SFDA.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising

Abstract

This work investigates Source-Free Domain Adaptation (SFDA), where a model adapts to a target domain without access to source data. A new augmentation technique, Shuffle PatchMix (SPM), and a novel reweighting strategy are introduced to enhance performance. SPM shuffles and blends image patches to generate diverse and challenging augmentations, while the reweighting strategy prioritizes reliable pseudo-labels to mitigate label noise. These techniques are particularly effective on smaller datasets like PACS, where overfitting and pseudo-label noise pose greater risks. State-of-the-art results are achieved on three major benchmarks: PACS, VisDA-C, and DomainNet-126. Notably, on PACS, improvements of 7.3% (79.4% to 86.7%) and 7.2% are observed in single-target and multi-target settings, respectively, while gains of 2.8% and 0.7% are attained on DomainNet-126 and VisDA-C. This combination of advanced augmentation and robust pseudo-label reweighting establishes a new benchmark for SFDA. The code is available at: https://github.com/PrasannaPulakurthi/SPM

Problem

• Enhance Source-Free Domain Adaptation without source data access
• Mitigate label noise using reliable pseudo-label reweighting strategy
• Improve performance on small datasets prone to overfitting

Innovation

• Shuffle PatchMix augmentation for diverse images
• Confidence-margin weighted pseudo-labels strategy
• Robust SFDA performance on small datasets