🤖 AI Summary
Dynamic distribution shifts between pretraining domains (e.g., ImageNet) and source/target domains significantly degrade performance in unsupervised domain adaptation (UDA) and source-free UDA (SFUDA), exacerbated by a detrimental “spontaneous feature contraction” effect during fine-tuning that erodes semantic structure. Method: To address this, we explicitly model the pretraining domain as a third domain and propose TriDA—a tri-domain collaborative framework. TriDA formulates a game-theoretic optimization objective grounded in gradient discrepancy theory, jointly enforcing semantic structural consistency and adversarial feature alignment to actively preserve pretraining knowledge. Contribution/Results: TriDA achieves state-of-the-art performance across multiple UDA and SFUDA benchmarks, yielding substantial gains in target-domain accuracy and enhanced generalization robustness.
📝 Abstract
Unsupervised domain adaptation(UDA) and Source-free UDA(SFUDA) methods formulate the problem involving two domains: source and target. They typically employ a standard training approach that begins with models pre-trained on large-scale datasets e.g., ImageNet, while rarely discussing its effect. Recognizing this gap, we investigate the following research questions: (1) What is the correlation among ImageNet, the source, and the target domain? (2) How does pre-training on ImageNet influence the target risk? To answer the first question, we empirically observed an interesting Spontaneous Pulling (SP) Effect in fine-tuning where the discrepancies between any two of the three domains (ImageNet, Source, Target) decrease but at the cost of the impaired semantic structure of the pre-train domain. For the second question, we put forward a theory to explain SP and quantify that the target risk is bound by gradient disparities among the three domains. Our observations reveal a key limitation of existing methods: it hinders the adaptation performance if the semantic cluster structure of the pre-train dataset (i.e.ImageNet) is impaired. To address it, we incorporate ImageNet as the third domain and redefine the UDA/SFUDA as a three-player game. Specifically, inspired by the theory and empirical findings, we present a novel framework termed TriDA which additionally preserves the semantic structure of the pre-train dataset during fine-tuning. Experimental results demonstrate that it achieves state-of-the-art performance across various UDA and SFUDA benchmarks.
